{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "P-pSKLNTR8xP"
   },
   "source": [
    "![JohnSnowLabs](https://nlp.johnsnowlabs.com/assets/images/logo.png)\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JohnSnowLabs/nlu/blob/master/examples/colab/healthcare/medical_named_entity_recognition/overview_medical_entity_recognizers.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e0-UfBGfyV6x"
   },
   "source": [
    "All the models avaiable are :\n",
    "\n",
    "\n",
    "\n",
    "| Language | nlp.load() reference                                         | Spark NLP Model reference          |\n",
    "| -------- | ------------------------------------------------------------ | ------------------------------------------------------------ |\n",
    "|English  | [med_ner.ade.clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_clinical_en.html) | [ner_ade_clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_clinical_en.html)                   |\n",
    "| English  | [med_ner.ade.clinical_bert](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_clinicalbert_en.html) | [ner_ade_clinicalbert](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_clinicalbert_en.html)                   |\n",
    "| English  | [med_ner.ade.ade_healthcare](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_healthcare_en.html) | [ner_ade_healthcare](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_healthcare_en.html)                   |\n",
    "| English  | [med_ner.anatomy](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_en.html) | [ner_anatomy](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_en.html)                   |\n",
    "| English  | [med_ner.anatomy.biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_anatomy_biobert_en.html) | [ner_anatomy_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_anatomy_biobert_en.html)                   |\n",
    "| English  | [med_ner.anatomy.coarse](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_coarse_en.html) | [ner_anatomy_coarse](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_coarse_en.html)                   |\n",
    "| English  | [med_ner.anatomy.coarse_biobert](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_coarse_biobert_en.html) | [ner_anatomy_coarse_biobert](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_coarse_biobert_en.html)                   |\n",
    "| English  | [med_ner.aspect_sentiment](https://nlp.johnsnowlabs.com/2021/03/31/ner_aspect_based_sentiment_en.html) | [ner_aspect_based_sentiment](https://nlp.johnsnowlabs.com/2021/03/31/ner_aspect_based_sentiment_en.html)                   |\n",
    "| English  | [med_ner.bacterial_species](https://nlp.johnsnowlabs.com/2021/04/01/ner_bacterial_species_en.html) | [ner_bacterial_species](https://nlp.johnsnowlabs.com/2021/04/01/ner_bacterial_species_en.html)                   |\n",
    "| English  | [med_ner.bionlp](https://nlp.johnsnowlabs.com/2021/03/31/ner_bionlp_en.html) | [ner_bionlp](https://nlp.johnsnowlabs.com/2021/03/31/ner_bionlp_en.html)                   |\n",
    "| English  | [med_ner.bionlp.biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_bionlp_biobert_en.html) | [ner_bionlp_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_bionlp_biobert_en.html)                   |\n",
    "| English  | [med_ner.cancer](https://nlp.johnsnowlabs.com/2021/03/31/ner_cancer_genetics_en.html) | [ner_cancer_genetics](https://nlp.johnsnowlabs.com/2021/03/31/ner_cancer_genetics_en.html)                   |\n",
    "| Englishs | [med_ner.cellular](https://nlp.johnsnowlabs.com/2021/03/31/ner_cellular_en.html) | [ner_cellular](https://nlp.johnsnowlabs.com/2021/03/31/ner_cellular_en.html)                   |\n",
    "| English  | [med_ner.cellular.biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_cellular_biobert_en.html) | [ner_cellular_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_cellular_biobert_en.html)                   |\n",
    "| English  | [med_ner.chemicals](https://nlp.johnsnowlabs.com/2021/04/01/ner_chemicals_en.html) | [ner_chemicals](https://nlp.johnsnowlabs.com/2021/04/01/ner_chemicals_en.html)                   |\n",
    "| English  | [med_ner.chemprot](https://nlp.johnsnowlabs.com/2021/04/01/ner_chemprot_biobert_en.html) | [ner_chemprot_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_chemprot_biobert_en.html)           |\n",
    "| English  | [med_ner.chemprot.clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_chemprot_clinical_en.html) | [ner_chemprot_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_chemprot_clinical_en.html)           |\n",
    "| English  | [med_ner.clinical](https://nlp.johnsnowlabs.com/2020/01/30/ner_clinical_en.html) | [ner_clinical](https://nlp.johnsnowlabs.com/2020/01/30/ner_clinical_en.html)           |\n",
    "| English  | [med_ner.clinical.biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_clinical_biobert_en.html) | [ner_clinical_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_clinical_biobert_en.html)           |\n",
    "| English  | med_ner.clinical.noncontrib | ner_clinical_noncontrib          |\n",
    "| English  | [med_ner.diseases](https://nlp.johnsnowlabs.com/2021/03/31/ner_diseases_en.html) | [ner_diseases](https://nlp.johnsnowlabs.com/2021/03/31/ner_diseases_en.html)           |\n",
    "| English  | [med_ner.diseases.biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_diseases_biobert_en.html) | [ner_diseases_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_diseases_biobert_en.html)           |\n",
    "| English  | [med_ner.diseases.large](https://nlp.johnsnowlabs.com/2021/04/01/ner_diseases_large_en.html) | [ner_diseases_large](https://nlp.johnsnowlabs.com/2021/04/01/ner_diseases_large_en.html)           |\n",
    "| English  | [med_ner.drugs](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_en.html) | [ner_drugs](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_en.html)           |\n",
    "| English  | [med_ner.drugsgreedy](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_greedy_en.html) | [ner_drugs_greedy](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_greedy_en.html)           |\n",
    "| English  | [med_ner.drugs.large](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_large_en.html) | [ner_drugs_large](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_large_en.html)           |\n",
    "| English  | [med_ner.events_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_events_biobert_en.html) | [ner_events_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_events_biobert_en.html)           |\n",
    "| English  | [med_ner.events_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_events_clinical_en.html) | [ner_events_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_events_clinical_en.html)           |\n",
    "| English  | [med_ner.events_healthcre](https://nlp.johnsnowlabs.com/2021/04/01/ner_events_healthcare_en.html) | [ner_events_healthcare](https://nlp.johnsnowlabs.com/2021/04/01/ner_events_healthcare_en.html)           |\n",
    "| English  | [med_ner.financial_contract](https://nlp.johnsnowlabs.com/2021/04/01/ner_financial_contract_en.html) | [ner_financial_contract](https://nlp.johnsnowlabs.com/2021/04/01/ner_financial_contract_en.html)           |\n",
    "| English  | [med_ner.healthcare](https://nlp.johnsnowlabs.com/2021/03/31/ner_healthcare_de.html) | [ner_healthcare](https://nlp.johnsnowlabs.com/2021/03/31/ner_healthcare_de.html)           |\n",
    "| English  | [med_ner.human_phenotype.gene_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_human_phenotype_gene_biobert_en.html) | [ner_human_phenotype_gene_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_human_phenotype_gene_biobert_en.html)           |\n",
    "| English  | [med_ner.human_phenotype.gene_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_human_phenotype_gene_clinical_en.html) | [ner_human_phenotype_gene_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_human_phenotype_gene_clinical_en.html)           |\n",
    "| English  | [med_ner.human_phenotype.go_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_human_phenotype_go_biobert_en.html) | [ner_human_phenotype_go_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_human_phenotype_go_biobert_en.html)           |\n",
    "| English  | [med_ner.human_phenotype.go_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_human_phenotype_go_clinical_en.html) | [ner_human_phenotype_go_clinical](https://nlp.johnsnowlabs.com/2021/03/31/ner_human_phenotype_go_clinical_en.html)           |\n",
    "| English  | [med_ner.jsl](https://nlp.johnsnowlabs.com/2021/03/31/ner_jsl_en.html) | [ner_jsl](https://nlp.johnsnowlabs.com/2021/03/31/ner_jsl_en.html)           |\n",
    "| English  | [med_ner.jsl.biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_jsl_biobert_en.html) | [ner_jsl_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_jsl_biobert_en.html)           |\n",
    "| English  | [med_ner.jsl.enriched](https://nlp.johnsnowlabs.com/2021/03/31/ner_jsl_enriched_en.html) | [ner_jsl_enriched](https://nlp.johnsnowlabs.com/2021/03/31/ner_jsl_enriched_en.html)           |\n",
    "| English  | [med_ner.jsl.enriched_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_jsl_enriched_biobert_en.html) | [ner_jsl_enriched_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_jsl_enriched_biobert_en.html)           |\n",
    "| English  | [med_ner.measurements](https://nlp.johnsnowlabs.com/2021/04/01/ner_measurements_clinical_en.html) | [ner_measurements_clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_measurements_clinical_en.html)           |\n",
    "| English  | [med_ner.medmentions](https://nlp.johnsnowlabs.com/2021/04/01/ner_medmentions_coarse_en.html) | [ner_medmentions_coarse](https://nlp.johnsnowlabs.com/2021/04/01/ner_medmentions_coarse_en.html)           |\n",
    "| English  | [med_ner.posology](https://nlp.johnsnowlabs.com/2020/04/15/ner_posology_en.html) | [ner_posology](https://nlp.johnsnowlabs.com/2020/04/15/ner_posology_en.html)           |\n",
    "| English  | [med_ner.posology.biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_biobert_en.html) | [ner_posology_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_biobert_en.html)           |\n",
    "| English  | [med_ner.posology.greedy](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_greedy_en.html) | [ner_posology_greedy](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_greedy_en.html)           |\n",
    "| English  | [med_ner.posology.healthcare](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_healthcare_en.html) | [ner_posology_healthcare](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_healthcare_en.html)           |\n",
    "| English  | [med_ner.posology.large](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_large_en.html) | [ner_posology_large](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_large_en.html)           |\n",
    "| English  | [med_ner.posology.large_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_large_biobert_en.html) | [ner_posology_large_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_large_biobert_en.html)           |\n",
    "| English  | [med_ner.posology.small](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_small_en.html) | [ner_posology_small](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_small_en.html)           |\n",
    "| English  | [med_ner.radiology](https://nlp.johnsnowlabs.com/2021/03/31/ner_radiology_en.html) | [ner_radiology](https://nlp.johnsnowlabs.com/2021/03/31/ner_radiology_en.html)           |\n",
    "| English  | [med_ner.radiology.wip_clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_radiology_wip_clinical_en.html) | [ner_radiology_wip_clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_radiology_wip_clinical_en.html)           |\n",
    "| English  | [med_ner.risk_factors](https://nlp.johnsnowlabs.com/2021/03/31/ner_risk_factors_en.html) | [ner_risk_factors](https://nlp.johnsnowlabs.com/2021/03/31/ner_risk_factors_en.html)           |\n",
    "| English  | [med_ner.risk_factors.biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_risk_factors_biobert_en.html) | [ner_risk_factors_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_risk_factors_biobert_en.html)           |\n",
    "| English  | med_ner.i2b2 | nerdl_i2b2          |\n",
    "| English  | [med_ner.tumour](https://nlp.johnsnowlabs.com/2021/04/01/nerdl_tumour_demo_en.html) | [nerdl_tumour_demo](https://nlp.johnsnowlabs.com/2021/04/01/nerdl_tumour_demo_en.html)           |\n",
    "| English  | med_ner.jsl.wip.clinical | jsl_ner_wip_clinical          |\n",
    "| English  | [med_ner.jsl.wip.clinical.greedy](https://nlp.johnsnowlabs.com/2021/03/31/jsl_ner_wip_clinical_en.html) | [jsl_ner_wip_greedy_clinical](https://nlp.johnsnowlabs.com/2021/03/31/jsl_ner_wip_clinical_en.html)           |\n",
    "| English  | [med_ner.jsl.wip.clinical.modifier](https://nlp.johnsnowlabs.com/2021/04/01/jsl_ner_wip_modifier_clinical_en.html) | [jsl_ner_wip_modifier_clinical](https://nlp.johnsnowlabs.com/2021/04/01/jsl_ner_wip_modifier_clinical_en.html)           |\n",
    "| English  | [med_ner.jsl.wip.clinical.rd](https://nlp.johnsnowlabs.com/2021/04/01/jsl_rd_ner_wip_greedy_clinical_en.html) | [jsl_rd_ner_wip_greedy_clinical](https://nlp.johnsnowlabs.com/2021/04/01/jsl_rd_ner_wip_greedy_clinical_en.html)           |\n",
    "| English  | [med_ner.deid.augmented](https://nlp.johnsnowlabs.com/2021/03/31/ner_deid_augmented_en.html) | [ner_deid_augmented](https://nlp.johnsnowlabs.com/2021/03/31/ner_deid_augmented_en.html)           |\n",
    "| English  | [med_ner.deid.biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_biobert_en.html) | [ner_deid_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_biobert_en.html)           |\n",
    "| English  | [med_ner.deid.enriched](https://nlp.johnsnowlabs.com/2021/03/31/ner_deid_enriched_en.html) | [ner_deid_enriched](https://nlp.johnsnowlabs.com/2021/03/31/ner_deid_enriched_en.html)           |\n",
    "| English  | [med_ner.deid.enriched_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_enriched_biobert_en.html) | [ner_deid_enriched_biobert](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_enriched_biobert_en.html)           |\n",
    "| English  | [med_ner.deid.large](https://nlp.johnsnowlabs.com/2021/03/31/ner_deid_large_en.html) | [ner_deid_large](https://nlp.johnsnowlabs.com/2021/03/31/ner_deid_large_en.html)           |\n",
    "| English  | [med_ner.deid.sd](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_sd_en.html) | [ner_deid_sd](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_sd_en.html)           |\n",
    "| English  | [med_ner.deid.sd_large](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_sd_large_en.html) | [ner_deid_sd_large](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_sd_large_en.html)           |\n",
    "| English  | med_ner.deid.synthetic | ner_deid_synthetic          |\n",
    "| English  | [med_ner.deid.dl](https://nlp.johnsnowlabs.com/2021/03/31/ner_deidentify_dl_en.html) | [ner_deidentify_dl](https://nlp.johnsnowlabs.com/2021/03/31/ner_deidentify_dl_en.html)           |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "BlfScWAZISSs",
    "ExecuteTime": {
     "end_time": "2024-08-11T07:48:24.156378700Z",
     "start_time": "2024-08-11T07:47:07.338269800Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001B[91m🚨 Your Spark-Healthcare is outdated, installed==5.4.0 but latest version==5.3.2\n",
      "You can run \u001B[92m nlp.install() \u001B[39mto update Spark-Healthcare\n",
      "📋 Loading license number 0 from C:\\Users\\gadde/.johnsnowlabs\\licenses/license_number_0_for_Spark-Healthcare_Spark-OCR.json\n",
      "👌 JSL-Home is up to date! \n",
      "Installing johnsnowlabs to C:\\Users\\gadde\\miniconda3\\envs\\Output_parser_cabir\\python.exe\n",
      "Installing pyspark to C:\\Users\\gadde\\miniconda3\\envs\\Output_parser_cabir\\python.exe\n",
      "Installing C:\\Users\\gadde/.johnsnowlabs\\py_installs\\spark_nlp_jsl-5.3.2-py3-none-any.whl to C:\\Users\\gadde\\miniconda3\\envs\\Output_parser_cabir\\python.exe\n",
      "Installed 3 products:\n",
      "🧪 John Snow Labs Python Library==5.3.6 installed! ✅ Easy access to all of John Snow Labs Enterprise Software! \n",
      "🐍+⚡ PySpark==3.4.0 installed! ✅ The big data Engine \n",
      "💊 Spark-Healthcare==5.3.2 installed! ✅ Heal the planet with NLP! \n",
      "\u001B[91m🚨 Your Spark-Healthcare is outdated, installed==5.3.2 but latest version==5.3.2\n",
      "You can run \u001B[92m nlp.install() \u001B[39mto update Spark-Healthcare\n"
     ]
    }
   ],
   "source": [
    "from johnsnowlabs import nlp\n",
    "\n",
    "# After uploading your license run this to install all licensed Python Wheels and pre-download Jars the Spark Session JVM\n",
    "nlp.install()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [Medical Named Entity Recognizer: ner_ade_clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_clinical_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 383
    },
    "id": "WOkulRB_Fj3T",
    "outputId": "0741af94-ee22-42d0-a314-e9eb8f33577a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_ade_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-ff37fce9-b6a6-47ec-83a8-d49fc208525c\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_ade</th>\n",
       "      <th>entities_ade_class</th>\n",
       "      <th>entities_ade_confidence</th>\n",
       "      <th>entities_ade_origin_chunk</th>\n",
       "      <th>entities_ade_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>Allegra</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.9612</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.4546172618865967, -0.3298020362854004, 0.7...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>redness</td>\n",
       "      <td>ADE</td>\n",
       "      <td>0.769</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.4546172618865967, -0.3298020362854004, 0.7...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>erythema</td>\n",
       "      <td>ADE</td>\n",
       "      <td>0.8092</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.4546172618865967, -0.3298020362854004, 0.7...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>swelling of the turbinates on the left</td>\n",
       "      <td>ADE</td>\n",
       "      <td>0.67710006</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.4546172618865967, -0.3298020362854004, 0.7...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ff37fce9-b6a6-47ec-83a8-d49fc208525c')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-ff37fce9-b6a6-47ec-83a8-d49fc208525c button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-ff37fce9-b6a6-47ec-83a8-d49fc208525c');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-605d1fe5-b792-48b6-b93a-d5a3a8be83a1\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-605d1fe5-b792-48b6-b93a-d5a3a8be83a1')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-605d1fe5-b792-48b6-b93a-d5a3a8be83a1 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  This is an 11-year-old female who comes in for...   \n",
       "0  This is an 11-year-old female who comes in for...   \n",
       "0  This is an 11-year-old female who comes in for...   \n",
       "0  This is an 11-year-old female who comes in for...   \n",
       "\n",
       "                             entities_ade entities_ade_class  \\\n",
       "0                                 Allegra               DRUG   \n",
       "0                                 redness                ADE   \n",
       "0                                erythema                ADE   \n",
       "0  swelling of the turbinates on the left                ADE   \n",
       "\n",
       "  entities_ade_confidence entities_ade_origin_chunk  \\\n",
       "0                  0.9612                         0   \n",
       "0                   0.769                         1   \n",
       "0                  0.8092                         2   \n",
       "0              0.67710006                         3   \n",
       "\n",
       "  entities_ade_origin_sentence  \\\n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.4546172618865967, -0.3298020362854004, 0.7...  \n",
       "0  [[0.4546172618865967, -0.3298020362854004, 0.7...  \n",
       "0  [[0.4546172618865967, -0.3298020362854004, 0.7...  \n",
       "0  [[0.4546172618865967, -0.3298020362854004, 0.7...  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.ade.clinical').predict('This is an 11-year-old female who comes in for two different things. 1. She was seen by the allergist. No allergies present, so she stopped her Allegra, but she is still real congested and does a lot of snorting. They do not notice a lot of snoring at night though, but she seems to be always like that. 2. On her right great toe, she has got some redness and erythema. Her skin is kind of peeling a little bit, but it has been like that for about a week and a half now. General: Well-developed female, in no acute distress, afebrile. HEENT: Sclerae and conjunctivae clear. Extraocular muscles intact. TMs clear. Nares patent. A little bit of swelling of the turbinates on the left. Oropharynx is essentially clear. Mucous membranes are moist. Neck: No lymphadenopathy. Chest: Clear. Abdomen: Positive bowel sounds and soft. Dermatologic: She has got redness along the lateral portion of her right great toe, but no bleeding or oozing. Some dryness of her skin. Her toenails themselves are very short and even on her left foot and her left great toe the toenails are very short.', output_level =  \"chunk\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "idJSVoIv8PWN"
   },
   "source": [
    "# [Medical Named Entity Recognizer: ner_ade_clinical](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_clinical_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 282
    },
    "id": "tJ9K9H277U6M",
    "outputId": "533b774f-7f01-4890-9cdf-0d1e45ace3bf"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_ade_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-1d1bf9df-0fd9-4862-af47-2706fd4745fe\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_ade</th>\n",
       "      <th>entities_ade_class</th>\n",
       "      <th>entities_ade_confidence</th>\n",
       "      <th>entities_ade_origin_chunk</th>\n",
       "      <th>entities_ade_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I feel a bit drowsy &amp; have a little blurred vi...</td>\n",
       "      <td>bit drowsy</td>\n",
       "      <td>ADE</td>\n",
       "      <td>0.46100003</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09337666630744934, 0.031265825033187866, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I feel a bit drowsy &amp; have a little blurred vi...</td>\n",
       "      <td>little blurred vision</td>\n",
       "      <td>ADE</td>\n",
       "      <td>0.6924</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09337666630744934, 0.031265825033187866, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I feel a bit drowsy &amp; have a little blurred vi...</td>\n",
       "      <td>insulin</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.9839</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09337666630744934, 0.031265825033187866, 0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1d1bf9df-0fd9-4862-af47-2706fd4745fe')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-1d1bf9df-0fd9-4862-af47-2706fd4745fe button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-1d1bf9df-0fd9-4862-af47-2706fd4745fe');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-9c2b06cd-2990-476a-9abe-f3b7860caf85\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-9c2b06cd-2990-476a-9abe-f3b7860caf85')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-9c2b06cd-2990-476a-9abe-f3b7860caf85 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document           entities_ade  \\\n",
       "0  I feel a bit drowsy & have a little blurred vi...             bit drowsy   \n",
       "0  I feel a bit drowsy & have a little blurred vi...  little blurred vision   \n",
       "0  I feel a bit drowsy & have a little blurred vi...                insulin   \n",
       "\n",
       "  entities_ade_class entities_ade_confidence entities_ade_origin_chunk  \\\n",
       "0                ADE              0.46100003                         0   \n",
       "0                ADE                  0.6924                         1   \n",
       "0               DRUG                  0.9839                         2   \n",
       "\n",
       "  entities_ade_origin_sentence  \\\n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.09337666630744934, 0.031265825033187866, 0...  \n",
       "0  [[0.09337666630744934, 0.031265825033187866, 0...  \n",
       "0  [[0.09337666630744934, 0.031265825033187866, 0...  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.ade.clinical').predict('I feel a bit drowsy & have a little blurred vision after taking an insulin', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NmBvudTc8ORb"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner_ade_clinicalbert\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_ade_clinicalbert_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 282
    },
    "id": "ihGmj4-7_WBm",
    "outputId": "abd3987f-b161-47cb-d48e-fe5953fd1106"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_ade_clinicalbert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_clinical_base_cased download started this may take some time.\n",
      "Approximate size to download 386.6 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-b9d72556-c1ae-401a-a223-338a4f970120\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_ade</th>\n",
       "      <th>entities_ade_class</th>\n",
       "      <th>entities_ade_confidence</th>\n",
       "      <th>entities_ade_origin_chunk</th>\n",
       "      <th>entities_ade_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I feel a bit drowsy &amp; have a little blurred vi...</td>\n",
       "      <td>drowsy</td>\n",
       "      <td>ADE</td>\n",
       "      <td>0.5249</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.045068055391311646, 0.35393112897872925, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I feel a bit drowsy &amp; have a little blurred vi...</td>\n",
       "      <td>blurred vision</td>\n",
       "      <td>ADE</td>\n",
       "      <td>0.8377</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.045068055391311646, 0.35393112897872925, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I feel a bit drowsy &amp; have a little blurred vi...</td>\n",
       "      <td>an insulin</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.7111</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.045068055391311646, 0.35393112897872925, ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b9d72556-c1ae-401a-a223-338a4f970120')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-b9d72556-c1ae-401a-a223-338a4f970120 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-b9d72556-c1ae-401a-a223-338a4f970120');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-d58f43f5-b3a8-4ee4-a8f6-e1e752198aac\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d58f43f5-b3a8-4ee4-a8f6-e1e752198aac')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-d58f43f5-b3a8-4ee4-a8f6-e1e752198aac button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document    entities_ade  \\\n",
       "0  I feel a bit drowsy & have a little blurred vi...          drowsy   \n",
       "0  I feel a bit drowsy & have a little blurred vi...  blurred vision   \n",
       "0  I feel a bit drowsy & have a little blurred vi...      an insulin   \n",
       "\n",
       "  entities_ade_class entities_ade_confidence entities_ade_origin_chunk  \\\n",
       "0                ADE                  0.5249                         0   \n",
       "0                ADE                  0.8377                         1   \n",
       "0               DRUG                  0.7111                         2   \n",
       "\n",
       "  entities_ade_origin_sentence  \\\n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-0.045068055391311646, 0.35393112897872925, ...  \n",
       "0  [[-0.045068055391311646, 0.35393112897872925, ...  \n",
       "0  [[-0.045068055391311646, 0.35393112897872925, ...  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.ade.clinical_bert').predict('I feel a bit drowsy & have a little blurred vision after taking an insulin', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3UXrXBz0BuoN"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner_ade_anatomy\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 237
    },
    "id": "eYhNblKOB8VM",
    "outputId": "43489708-4f3c-41e0-be64-6c0cb308686e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_anatomy download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-8025c6f9-248c-4805-9934-4a530a5503aa\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_anatomy</th>\n",
       "      <th>entities_anatomy_class</th>\n",
       "      <th>entities_anatomy_confidence</th>\n",
       "      <th>entities_anatomy_origin_chunk</th>\n",
       "      <th>entities_anatomy_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>skin</td>\n",
       "      <td>Organ</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.4546172618865967, -0.3298020362854004, 0.7...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8025c6f9-248c-4805-9934-4a530a5503aa')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-8025c6f9-248c-4805-9934-4a530a5503aa button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-8025c6f9-248c-4805-9934-4a530a5503aa');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_anatomy  \\\n",
       "0  This is an 11-year-old female who comes in for...             skin   \n",
       "\n",
       "  entities_anatomy_class entities_anatomy_confidence  \\\n",
       "0                  Organ                         1.0   \n",
       "\n",
       "  entities_anatomy_origin_chunk entities_anatomy_origin_sentence  \\\n",
       "0                             0                                0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.4546172618865967, -0.3298020362854004, 0.7...  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.anatomy').predict('This is an 11-year-old female who comes in for two different things. 1. She was seen by the allergist. No allergies present, so she stopped her Allegra, but she is still real congested and does a lot of snorting. They do not notice a lot of snoring at night though, but she seems to be always like that. 2. On her right great toe, she has got some redness and erythema. Her skin is kind of peeling a little bit, but it has been like that for about a week and a half now.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HSPfCm_MCbNs"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner_ade_anatomy.biobert\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_anatomy_biobert_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 334
    },
    "id": "olyg3I_zClGV",
    "outputId": "5017dd27-8017-4f72-93b8-9a18ca74a40a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_anatomy_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-4b2e738d-cd5d-4e19-b1e9-fd10ab567335\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_anatomy</th>\n",
       "      <th>entities_anatomy_class</th>\n",
       "      <th>entities_anatomy_confidence</th>\n",
       "      <th>entities_anatomy_origin_chunk</th>\n",
       "      <th>entities_anatomy_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>right</td>\n",
       "      <td>Organism_subdivision</td>\n",
       "      <td>0.9953</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5449297428131104, -0.1307530254125595, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>toe</td>\n",
       "      <td>Organism_subdivision</td>\n",
       "      <td>0.9972</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5449297428131104, -0.1307530254125595, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>skin</td>\n",
       "      <td>Organ</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5449297428131104, -0.1307530254125595, -0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4b2e738d-cd5d-4e19-b1e9-fd10ab567335')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-4b2e738d-cd5d-4e19-b1e9-fd10ab567335 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-4b2e738d-cd5d-4e19-b1e9-fd10ab567335');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-404251ed-f090-445d-80a4-7beedb5418ab\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-404251ed-f090-445d-80a4-7beedb5418ab')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-404251ed-f090-445d-80a4-7beedb5418ab button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_anatomy  \\\n",
       "0  This is an 11-year-old female who comes in for...            right   \n",
       "0  This is an 11-year-old female who comes in for...              toe   \n",
       "0  This is an 11-year-old female who comes in for...             skin   \n",
       "\n",
       "  entities_anatomy_class entities_anatomy_confidence  \\\n",
       "0   Organism_subdivision                      0.9953   \n",
       "0   Organism_subdivision                      0.9972   \n",
       "0                  Organ                         1.0   \n",
       "\n",
       "  entities_anatomy_origin_chunk entities_anatomy_origin_sentence  \\\n",
       "0                             0                                0   \n",
       "0                             1                                0   \n",
       "0                             2                                0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-0.5449297428131104, -0.1307530254125595, -0...  \n",
       "0  [[-0.5449297428131104, -0.1307530254125595, -0...  \n",
       "0  [[-0.5449297428131104, -0.1307530254125595, -0...  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.anatomy.biobert').predict('This is an 11-year-old female who comes in for two different things. 1. She was seen by the allergist. No allergies present, so she stopped her Allegra, but she is still real congested and does a lot of snorting. They do not notice a lot of snoring at night though, but she seems to be always like that. 2. On her right great toe, she has got some redness and erythema. Her skin is kind of peeling a little bit, but it has been like that for about a week and a half now.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZzKJj-s2DjFJ"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner_ade_anatomy.coarse.biobert\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_anatomy_coarse_biobert_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 286
    },
    "id": "KVeN68G_DryT",
    "outputId": "0d38fdb1-0aa2-4b1e-a4c3-999de2a1c241"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_anatomy_coarse_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-f1b4d5d8-1c73-4347-9faf-5d2f3fea7292\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_anatomy</th>\n",
       "      <th>entities_anatomy_class</th>\n",
       "      <th>entities_anatomy_confidence</th>\n",
       "      <th>entities_anatomy_origin_chunk</th>\n",
       "      <th>entities_anatomy_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>right great toe</td>\n",
       "      <td>Anatomy</td>\n",
       "      <td>0.8986667</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5449297428131104, -0.1307530254125595, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>This is an 11-year-old female who comes in for...</td>\n",
       "      <td>skin</td>\n",
       "      <td>Anatomy</td>\n",
       "      <td>0.7334</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5449297428131104, -0.1307530254125595, -0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f1b4d5d8-1c73-4347-9faf-5d2f3fea7292')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-f1b4d5d8-1c73-4347-9faf-5d2f3fea7292 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-f1b4d5d8-1c73-4347-9faf-5d2f3fea7292');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-84a050be-6a0f-423e-b216-af97ebfcdfdf\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-84a050be-6a0f-423e-b216-af97ebfcdfdf')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-84a050be-6a0f-423e-b216-af97ebfcdfdf button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_anatomy  \\\n",
       "0  This is an 11-year-old female who comes in for...  right great toe   \n",
       "0  This is an 11-year-old female who comes in for...             skin   \n",
       "\n",
       "  entities_anatomy_class entities_anatomy_confidence  \\\n",
       "0                Anatomy                   0.8986667   \n",
       "0                Anatomy                      0.7334   \n",
       "\n",
       "  entities_anatomy_origin_chunk entities_anatomy_origin_sentence  \\\n",
       "0                             0                                0   \n",
       "0                             1                                0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-0.5449297428131104, -0.1307530254125595, -0...  \n",
       "0  [[-0.5449297428131104, -0.1307530254125595, -0...  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.anatomy.coarse_biobert').predict('This is an 11-year-old female who comes in for two different things. 1. She was seen by the allergist. No allergies present, so she stopped her Allegra, but she is still real congested and does a lot of snorting. They do not notice a lot of snoring at night though, but she seems to be always like that. 2. On her right great toe, she has got some redness and erythema. Her skin is kind of peeling a little bit, but it has been like that for about a week and a half now.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fA-AbJkUEbdc"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  aspect_sentiment\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_aspect_based_sentiment_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 712
    },
    "id": "-axw8-lYE17g",
    "outputId": "2051268f-09b6-4f4a-e137-fd864dbea53b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_aspect_based_sentiment download started this may take some time.\n",
      "[OK!]\n",
      "glove_6B_300 download started this may take some time.\n",
      "Approximate size to download 426.2 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-38b719c7-7642-4c08-800e-eb5d803381bf\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_aspect_sentiment</th>\n",
       "      <th>entities_aspect_sentiment_class</th>\n",
       "      <th>entities_aspect_sentiment_confidence</th>\n",
       "      <th>entities_aspect_sentiment_origin_chunk</th>\n",
       "      <th>entities_aspect_sentiment_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>Thai-style main</td>\n",
       "      <td>POS</td>\n",
       "      <td>0.849</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>flavours</td>\n",
       "      <td>POS</td>\n",
       "      <td>0.8488</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>vegetarian</td>\n",
       "      <td>POS</td>\n",
       "      <td>0.995</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>service</td>\n",
       "      <td>NEG</td>\n",
       "      <td>0.9952</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>chips</td>\n",
       "      <td>NEG</td>\n",
       "      <td>0.9955</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-38b719c7-7642-4c08-800e-eb5d803381bf')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-38b719c7-7642-4c08-800e-eb5d803381bf button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-38b719c7-7642-4c08-800e-eb5d803381bf');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-bc3da898-e264-43da-8154-c6819751397c\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-bc3da898-e264-43da-8154-c6819751397c')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-bc3da898-e264-43da-8154-c6819751397c button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "\n",
       "  entities_aspect_sentiment entities_aspect_sentiment_class  \\\n",
       "0           Thai-style main                             POS   \n",
       "0                  flavours                             POS   \n",
       "0                vegetarian                             POS   \n",
       "0                   service                             NEG   \n",
       "0                     chips                             NEG   \n",
       "\n",
       "  entities_aspect_sentiment_confidence entities_aspect_sentiment_origin_chunk  \\\n",
       "0                                0.849                                      0   \n",
       "0                               0.8488                                      1   \n",
       "0                                0.995                                      2   \n",
       "0                               0.9952                                      3   \n",
       "0                               0.9955                                      4   \n",
       "\n",
       "  entities_aspect_sentiment_origin_sentence  \\\n",
       "0                                         0   \n",
       "0                                         0   \n",
       "0                                         0   \n",
       "0                                         0   \n",
       "0                                         0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.aspect_sentiment').predict('Came for lunch my sister. We loved our Thai-style main which amazing with lots of flavours very impressive for vegetarian. But the service was below average and the chips were too terrible to finish.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 712
    },
    "id": "MZXl1fQdin4n",
    "outputId": "443df274-471d-45cc-d6ac-cef4a5f136c9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_aspect_based_sentiment download started this may take some time.\n",
      "[OK!]\n",
      "glove_6B_300 download started this may take some time.\n",
      "Approximate size to download 426.2 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-cb2fa198-0f27-481b-9af4-691d5e2009ff\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_aspect_sentiment</th>\n",
       "      <th>entities_aspect_sentiment_class</th>\n",
       "      <th>entities_aspect_sentiment_confidence</th>\n",
       "      <th>entities_aspect_sentiment_origin_chunk</th>\n",
       "      <th>entities_aspect_sentiment_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>Thai-style main</td>\n",
       "      <td>POS</td>\n",
       "      <td>0.849</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>flavours</td>\n",
       "      <td>POS</td>\n",
       "      <td>0.8488</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>vegetarian</td>\n",
       "      <td>POS</td>\n",
       "      <td>0.995</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>service</td>\n",
       "      <td>NEG</td>\n",
       "      <td>0.9952</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Came for lunch my sister. We loved our Thai-st...</td>\n",
       "      <td>chips</td>\n",
       "      <td>NEG</td>\n",
       "      <td>0.9955</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.15535999834537506, -0.0022360999137163162,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-cb2fa198-0f27-481b-9af4-691d5e2009ff')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-cb2fa198-0f27-481b-9af4-691d5e2009ff button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-cb2fa198-0f27-481b-9af4-691d5e2009ff');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-103405cf-31eb-41a3-94a2-f7898f74591e\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-103405cf-31eb-41a3-94a2-f7898f74591e')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-103405cf-31eb-41a3-94a2-f7898f74591e button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "0  Came for lunch my sister. We loved our Thai-st...   \n",
       "\n",
       "  entities_aspect_sentiment entities_aspect_sentiment_class  \\\n",
       "0           Thai-style main                             POS   \n",
       "0                  flavours                             POS   \n",
       "0                vegetarian                             POS   \n",
       "0                   service                             NEG   \n",
       "0                     chips                             NEG   \n",
       "\n",
       "  entities_aspect_sentiment_confidence entities_aspect_sentiment_origin_chunk  \\\n",
       "0                                0.849                                      0   \n",
       "0                               0.8488                                      1   \n",
       "0                                0.995                                      2   \n",
       "0                               0.9952                                      3   \n",
       "0                               0.9955                                      4   \n",
       "\n",
       "  entities_aspect_sentiment_origin_sentence  \\\n",
       "0                                         0   \n",
       "0                                         0   \n",
       "0                                         0   \n",
       "0                                         0   \n",
       "0                                         0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  \n",
       "0  [[0.15535999834537506, -0.0022360999137163162,...  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.aspect_sentiment').predict('Came for lunch my sister. We loved our Thai-style main which amazing with lots of flavours very impressive for vegetarian. But the service was below average and the chips were too terrible to finish.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9XDen3uuHMcB"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.bionlp\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_bionlp_en.html)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 675
    },
    "id": "PyoMaxnjHXha",
    "outputId": "ec7b546d-146b-492d-c597-f43e20095fe0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_bionlp download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-3488db57-b58a-4f02-9cab-074a0682c81b\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_bionlp</th>\n",
       "      <th>entities_bionlp_class</th>\n",
       "      <th>entities_bionlp_confidence</th>\n",
       "      <th>entities_bionlp_origin_chunk</th>\n",
       "      <th>entities_bionlp_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>human</td>\n",
       "      <td>Organism</td>\n",
       "      <td>0.9997</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>Kir 3.3</td>\n",
       "      <td>Gene_or_gene_product</td>\n",
       "      <td>0.9972</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>GIRK3</td>\n",
       "      <td>Gene_or_gene_product</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>potassium</td>\n",
       "      <td>Simple_chemical</td>\n",
       "      <td>0.5571</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>GIRK</td>\n",
       "      <td>Gene_or_gene_product</td>\n",
       "      <td>0.9989</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>chromosome 1q21-23</td>\n",
       "      <td>Cellular_component</td>\n",
       "      <td>0.7428</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>pancreas</td>\n",
       "      <td>Organ</td>\n",
       "      <td>0.9993</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>tissues</td>\n",
       "      <td>Tissue</td>\n",
       "      <td>0.9754</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>fat andskeletal muscle</td>\n",
       "      <td>Tissue</td>\n",
       "      <td>0.97853327</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>Type II</td>\n",
       "      <td>Gene_or_gene_product</td>\n",
       "      <td>0.98995</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3488db57-b58a-4f02-9cab-074a0682c81b')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-3488db57-b58a-4f02-9cab-074a0682c81b button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-3488db57-b58a-4f02-9cab-074a0682c81b');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-213c27e8-a3e1-4a55-9213-99fc09bc9f0a\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-213c27e8-a3e1-4a55-9213-99fc09bc9f0a')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-213c27e8-a3e1-4a55-9213-99fc09bc9f0a button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document         entities_bionlp  \\\n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...                   human   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...                 Kir 3.3   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...                   GIRK3   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...               potassium   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...                    GIRK   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...      chromosome 1q21-23   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...                pancreas   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...                 tissues   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...  fat andskeletal muscle   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...                 Type II   \n",
       "\n",
       "  entities_bionlp_class entities_bionlp_confidence  \\\n",
       "0              Organism                     0.9997   \n",
       "0  Gene_or_gene_product                     0.9972   \n",
       "0  Gene_or_gene_product                        1.0   \n",
       "0       Simple_chemical                     0.5571   \n",
       "0  Gene_or_gene_product                     0.9989   \n",
       "0    Cellular_component                     0.7428   \n",
       "0                 Organ                     0.9993   \n",
       "0                Tissue                     0.9754   \n",
       "0                Tissue                 0.97853327   \n",
       "0  Gene_or_gene_product                    0.98995   \n",
       "\n",
       "  entities_bionlp_origin_chunk entities_bionlp_origin_sentence  \\\n",
       "0                            0                               0   \n",
       "0                            1                               0   \n",
       "0                            2                               0   \n",
       "0                            3                               0   \n",
       "0                            4                               0   \n",
       "0                            5                               0   \n",
       "0                            6                               0   \n",
       "0                            7                               0   \n",
       "0                            8                               0   \n",
       "0                            9                               0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.bionlp').predict('The human KCNJ9 (Kir 3.3, GIRK3) is a member of the G-protein-activated inwardly rectifying potassium (GIRK) channel family. Here we describe the genomicorganization of the KCNJ9 locus on chromosome 1q21-23 as a candidate gene forType II diabetes mellitus in the Pima Indian population. The gene spansapproximately 7.6 kb and contains one noncoding and two coding exons separated byapproximately 2.2 and approximately 2.6 kb introns, respectively. We identified14 single nucleotide polymorphisms (SNPs), including one that predicts aVal366Ala substitution, and an 8 base-pair (bp) insertion/deletion. Ourexpression studies revealed the presence of the transcript in various humantissues including pancreas, and two major insulin-responsive tissues: fat andskeletal muscle. The characterization of the KCNJ9 gene should facilitate furtherstudies on the function of the KCNJ9 protein and allow evaluation of thepotential role of the locus in Type II diabetes.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "T1cpMQzvH2oi"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.bionlp.biobert\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_bionlp_biobert_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 481
    },
    "id": "UgSk0ILUIArw",
    "outputId": "f166ab44-eed0-4801-c0e8-a3401f0fc919"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_bionlp_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-b6eda5f5-2cbc-452e-9ab1-75ef2beb47cd\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_bionlp</th>\n",
       "      <th>entities_bionlp_class</th>\n",
       "      <th>entities_bionlp_confidence</th>\n",
       "      <th>entities_bionlp_origin_chunk</th>\n",
       "      <th>entities_bionlp_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>human</td>\n",
       "      <td>Organism</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.6279052495956421, -0.10282155871391296, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>KCNJ9</td>\n",
       "      <td>Gene_or_gene_product</td>\n",
       "      <td>0.9999</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.6279052495956421, -0.10282155871391296, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>Kir 3.3</td>\n",
       "      <td>Gene_or_gene_product</td>\n",
       "      <td>0.99995</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.6279052495956421, -0.10282155871391296, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>GIRK3</td>\n",
       "      <td>Gene_or_gene_product</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.6279052495956421, -0.10282155871391296, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>GIRK) channel</td>\n",
       "      <td>Gene_or_gene_product</td>\n",
       "      <td>0.75189996</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.6279052495956421, -0.10282155871391296, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>KCNJ9</td>\n",
       "      <td>Gene_or_gene_product</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.6279052495956421, -0.10282155871391296, -...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b6eda5f5-2cbc-452e-9ab1-75ef2beb47cd')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-b6eda5f5-2cbc-452e-9ab1-75ef2beb47cd button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-b6eda5f5-2cbc-452e-9ab1-75ef2beb47cd');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-4aadc4a7-6344-4daf-b7e1-4603ef3c47c4\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4aadc4a7-6344-4daf-b7e1-4603ef3c47c4')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-4aadc4a7-6344-4daf-b7e1-4603ef3c47c4 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_bionlp  \\\n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...           human   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...           KCNJ9   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...         Kir 3.3   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...           GIRK3   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   GIRK) channel   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...           KCNJ9   \n",
       "\n",
       "  entities_bionlp_class entities_bionlp_confidence  \\\n",
       "0              Organism                        1.0   \n",
       "0  Gene_or_gene_product                     0.9999   \n",
       "0  Gene_or_gene_product                    0.99995   \n",
       "0  Gene_or_gene_product                        1.0   \n",
       "0  Gene_or_gene_product                 0.75189996   \n",
       "0  Gene_or_gene_product                        1.0   \n",
       "\n",
       "  entities_bionlp_origin_chunk entities_bionlp_origin_sentence  \\\n",
       "0                            0                               0   \n",
       "0                            1                               0   \n",
       "0                            2                               0   \n",
       "0                            3                               0   \n",
       "0                            4                               0   \n",
       "0                            5                               0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-0.6279052495956421, -0.10282155871391296, -...  \n",
       "0  [[-0.6279052495956421, -0.10282155871391296, -...  \n",
       "0  [[-0.6279052495956421, -0.10282155871391296, -...  \n",
       "0  [[-0.6279052495956421, -0.10282155871391296, -...  \n",
       "0  [[-0.6279052495956421, -0.10282155871391296, -...  \n",
       "0  [[-0.6279052495956421, -0.10282155871391296, -...  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.bionlp.biobert').predict('The human KCNJ9 (Kir 3.3, GIRK3) is a member of the G-protein-activated inwardly rectifying potassium (GIRK) channel family. Here we describe the genomicorganization of the KCNJ9 locus on chromosome 1q21-23 as a candidate gene forType II diabetes mellitus in the Pima Indian population. The gene spansapproximately 7.6 kb and contains one noncoding and two coding exons separated byapproximately 2.2 and approximately 2.6 kb introns, respectively. We identified14 single nucleotide polymorphisms (SNPs), including one that predicts aVal366Ala substitution, and an 8 base-pair (bp) insertion/deletion. Ourexpression studies revealed the presence of the transcript in various humantissues including pancreas, and two major insulin-responsive tissues: fat andskeletal muscle. The characterization of the KCNJ9 gene should facilitate furtherstudies on the function of the KCNJ9 protein and allow evaluation of thepotential role of the locus in Type II diabetes.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nfQwWMBqIZ4B"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.cancer\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_cancer_genetics_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 822
    },
    "id": "OvVZre0zI-Zy",
    "outputId": "574babdb-baa0-4cb7-a822-65628e7d5d66"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_cancer_genetics download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-adf90900-4ed7-4c13-bad7-e7f0c21737e2\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_cancer</th>\n",
       "      <th>entities_cancer_class</th>\n",
       "      <th>entities_cancer_confidence</th>\n",
       "      <th>entities_cancer_origin_chunk</th>\n",
       "      <th>entities_cancer_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>human KCNJ9</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.53155</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>Kir 3.3</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.80649996</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>GIRK3</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.9192</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>G-protein-activated inwardly rectifying potass...</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.63857776</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>KCNJ9 locus</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.97065</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>chromosome 1q21-23</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.90345</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>candidate gene</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.71889997</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>coding exons</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.60580003</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>2.6 kb introns</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.8032333</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>nucleotide polymorphisms</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.70755</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>SNPs),</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.90715003</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>KCNJ9 gene</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.9683</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>KCNJ9 protein</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.74255</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-adf90900-4ed7-4c13-bad7-e7f0c21737e2')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-adf90900-4ed7-4c13-bad7-e7f0c21737e2 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-adf90900-4ed7-4c13-bad7-e7f0c21737e2');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-e90d6ac5-e78c-4a27-bbce-1ef82145e7b2\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e90d6ac5-e78c-4a27-bbce-1ef82145e7b2')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-e90d6ac5-e78c-4a27-bbce-1ef82145e7b2 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "\n",
       "                                     entities_cancer entities_cancer_class  \\\n",
       "0                                        human KCNJ9                   DNA   \n",
       "0                                            Kir 3.3               protein   \n",
       "0                                              GIRK3                   DNA   \n",
       "0  G-protein-activated inwardly rectifying potass...               protein   \n",
       "0                                        KCNJ9 locus                   DNA   \n",
       "0                                 chromosome 1q21-23                   DNA   \n",
       "0                                     candidate gene                   DNA   \n",
       "0                                       coding exons                   DNA   \n",
       "0                                     2.6 kb introns                   DNA   \n",
       "0                           nucleotide polymorphisms                   DNA   \n",
       "0                                             SNPs),                   DNA   \n",
       "0                                         KCNJ9 gene                   DNA   \n",
       "0                                      KCNJ9 protein               protein   \n",
       "\n",
       "  entities_cancer_confidence entities_cancer_origin_chunk  \\\n",
       "0                    0.53155                            0   \n",
       "0                 0.80649996                            1   \n",
       "0                     0.9192                            2   \n",
       "0                 0.63857776                            3   \n",
       "0                    0.97065                            4   \n",
       "0                    0.90345                            5   \n",
       "0                 0.71889997                            6   \n",
       "0                 0.60580003                            7   \n",
       "0                  0.8032333                            8   \n",
       "0                    0.70755                            9   \n",
       "0                 0.90715003                           10   \n",
       "0                     0.9683                           11   \n",
       "0                    0.74255                           12   \n",
       "\n",
       "  entities_cancer_origin_sentence  \\\n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "0                               0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.cancer').predict('The human KCNJ9 (Kir 3.3, GIRK3) is a member of the G-protein-activated inwardly rectifying potassium (GIRK) channel family. Here we describe the genomicorganization of the KCNJ9 locus on chromosome 1q21-23 as a candidate gene forType II diabetes mellitus in the Pima Indian population. The gene spansapproximately 7.6 kb and contains one noncoding and two coding exons separated byapproximately 2.2 and approximately 2.6 kb introns, respectively. We identified14 single nucleotide polymorphisms (SNPs), including one that predicts aVal366Ala substitution, and an 8 base-pair (bp) insertion/deletion. Ourexpression studies revealed the presence of the transcript in various humantissues including pancreas, and two major insulin-responsive tissues: fat andskeletal muscle. The characterization of the KCNJ9 gene should facilitate furtherstudies on the function of the KCNJ9 protein and allow evaluation of thepotential role of the locus in Type II diabetes.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MhSkvimKJTjB"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.cellular\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_cellular_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "B_wO9u0NJe0C",
    "outputId": "e5638b82-a749-496c-9876-18c3480d236a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_cellular download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-9af3356e-4158-4e9b-b1fc-8b32a13c70c6\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_cellular</th>\n",
       "      <th>entities_cellular_class</th>\n",
       "      <th>entities_cellular_confidence</th>\n",
       "      <th>entities_cellular_origin_chunk</th>\n",
       "      <th>entities_cellular_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>intracellular signaling proteins</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.78656673</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>human T-cell leukemia virus type 1 promoter</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.5614286</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>Tax</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.9874</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>Tax-responsive element 1</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.9712</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>cyclic AMP-responsive members</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.5933333</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>CREB/ATF family</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.79015</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>transcription factors</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.7412</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>Tax</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.9663</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>human T-cell leukemia virus type 1 Tax-respons...</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.69261116</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>TRE-1),</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.86385</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>yeast reporter strain</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.8071334</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>lacZ gene</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.96239996</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>CYC1 promoter</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.98785</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>TRE-1</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.9992</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>cyclic AMP response element-binding protein</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.73576</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>CREB</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.9947</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>CREB</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.9935</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>GAL4 activation domain</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.6394667</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>GAD</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.9932</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>reporter gene</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.81165004</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>Tax</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.938</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9af3356e-4158-4e9b-b1fc-8b32a13c70c6')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-9af3356e-4158-4e9b-b1fc-8b32a13c70c6 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-9af3356e-4158-4e9b-b1fc-8b32a13c70c6');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-89ccfba8-fea0-4382-9c1a-d49eb6c5e174\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-89ccfba8-fea0-4382-9c1a-d49eb6c5e174')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-89ccfba8-fea0-4382-9c1a-d49eb6c5e174 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "\n",
       "                                   entities_cellular entities_cellular_class  \\\n",
       "0                   intracellular signaling proteins                 protein   \n",
       "0        human T-cell leukemia virus type 1 promoter                     DNA   \n",
       "0                                                Tax                 protein   \n",
       "0                           Tax-responsive element 1                     DNA   \n",
       "0                      cyclic AMP-responsive members                     DNA   \n",
       "0                                    CREB/ATF family                 protein   \n",
       "0                              transcription factors                 protein   \n",
       "0                                                Tax                 protein   \n",
       "0  human T-cell leukemia virus type 1 Tax-respons...                     DNA   \n",
       "0                                            TRE-1),                     DNA   \n",
       "0                              yeast reporter strain                     DNA   \n",
       "0                                          lacZ gene                     DNA   \n",
       "0                                      CYC1 promoter                     DNA   \n",
       "0                                              TRE-1                     DNA   \n",
       "0        cyclic AMP response element-binding protein                 protein   \n",
       "0                                               CREB                 protein   \n",
       "0                                               CREB                 protein   \n",
       "0                             GAL4 activation domain                 protein   \n",
       "0                                                GAD                 protein   \n",
       "0                                      reporter gene                     DNA   \n",
       "0                                                Tax                 protein   \n",
       "\n",
       "  entities_cellular_confidence entities_cellular_origin_chunk  \\\n",
       "0                   0.78656673                              0   \n",
       "0                    0.5614286                              1   \n",
       "0                       0.9874                              2   \n",
       "0                       0.9712                              3   \n",
       "0                    0.5933333                              4   \n",
       "0                      0.79015                              5   \n",
       "0                       0.7412                              6   \n",
       "0                       0.9663                              7   \n",
       "0                   0.69261116                              8   \n",
       "0                      0.86385                              9   \n",
       "0                    0.8071334                             10   \n",
       "0                   0.96239996                             11   \n",
       "0                      0.98785                             12   \n",
       "0                       0.9992                             13   \n",
       "0                      0.73576                             14   \n",
       "0                       0.9947                             15   \n",
       "0                       0.9935                             16   \n",
       "0                    0.6394667                             17   \n",
       "0                       0.9932                             18   \n",
       "0                   0.81165004                             19   \n",
       "0                        0.938                             20   \n",
       "\n",
       "  entities_cellular_origin_sentence  \\\n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.cellular').predict('Detection of various other intracellular signaling proteins is also described. Genetic characterization of transactivation of the human T-cell leukemia virus type 1 promoter: Binding of Tax to Tax-responsive element 1 is mediated by the cyclic AMP-responsive members of the CREB/ATF family of transcription factors. To achieve a better understanding of the mechanism of transactivation by Tax of human T-cell leukemia virus type 1 Tax-responsive element 1 (TRE-1), we developed a genetic approach with Saccharomyces cerevisiae. We constructed a yeast reporter strain containing the lacZ gene under the control of the CYC1 promoter associated with three copies of TRE-1. Expression of either the cyclic AMP response element-binding protein (CREB) or CREB fused to the GAL4 activation domain (GAD) in this strain did not modify the expression of the reporter gene. Tax alone was also inactive.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bq1mzs1NJ4n0"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.cellular.biobert\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_cellular_biobert_en.html)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 773
    },
    "id": "h6emBCO0J_7m",
    "outputId": "2342524d-6bea-4847-f540-a5b1d634d5ea"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_cellular_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-fc4463be-7714-4d51-bde8-128a4ea17b64\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_cellular</th>\n",
       "      <th>entities_cellular_class</th>\n",
       "      <th>entities_cellular_confidence</th>\n",
       "      <th>entities_cellular_origin_chunk</th>\n",
       "      <th>entities_cellular_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>intracellular signaling proteins</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.6826666</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>human T-cell leukemia virus type 1 promoter</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.43541428</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>Tax</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.7016</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>Tax-responsive element 1</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.75670004</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>cyclic AMP-responsive members</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.61083335</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>CREB/ATF family</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.87955</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>transcription factors</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.7499</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>Tax</td>\n",
       "      <td>protein</td>\n",
       "      <td>0.8283</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>Tax-responsive element 1</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.6871667</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>TRE-1</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.9986</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>lacZ gene</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.606</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>CYC1</td>\n",
       "      <td>DNA</td>\n",
       "      <td>0.828</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fc4463be-7714-4d51-bde8-128a4ea17b64')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-fc4463be-7714-4d51-bde8-128a4ea17b64 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-fc4463be-7714-4d51-bde8-128a4ea17b64');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-57d6938f-88b7-417d-8b93-59f2ad288dd7\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-57d6938f-88b7-417d-8b93-59f2ad288dd7')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-57d6938f-88b7-417d-8b93-59f2ad288dd7 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "0  Detection of various other intracellular signa...   \n",
       "\n",
       "                             entities_cellular entities_cellular_class  \\\n",
       "0             intracellular signaling proteins                 protein   \n",
       "0  human T-cell leukemia virus type 1 promoter                     DNA   \n",
       "0                                          Tax                 protein   \n",
       "0                     Tax-responsive element 1                     DNA   \n",
       "0                cyclic AMP-responsive members                 protein   \n",
       "0                              CREB/ATF family                 protein   \n",
       "0                        transcription factors                 protein   \n",
       "0                                          Tax                 protein   \n",
       "0                     Tax-responsive element 1                     DNA   \n",
       "0                                        TRE-1                     DNA   \n",
       "0                                    lacZ gene                     DNA   \n",
       "0                                         CYC1                     DNA   \n",
       "\n",
       "  entities_cellular_confidence entities_cellular_origin_chunk  \\\n",
       "0                    0.6826666                              0   \n",
       "0                   0.43541428                              1   \n",
       "0                       0.7016                              2   \n",
       "0                   0.75670004                              3   \n",
       "0                   0.61083335                              4   \n",
       "0                      0.87955                              5   \n",
       "0                       0.7499                              6   \n",
       "0                       0.8283                              7   \n",
       "0                    0.6871667                              8   \n",
       "0                       0.9986                              9   \n",
       "0                        0.606                             10   \n",
       "0                        0.828                             11   \n",
       "\n",
       "  entities_cellular_origin_sentence  \\\n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.cellular.biobert').predict('Detection of various other intracellular signaling proteins is also described. Genetic characterization of transactivation of the human T-cell leukemia virus type 1 promoter: Binding of Tax to Tax-responsive element 1 is mediated by the cyclic AMP-responsive members of the CREB/ATF family of transcription factors. To achieve a better understanding of the mechanism of transactivation by Tax of human T-cell leukemia virus type 1 Tax-responsive element 1 (TRE-1), we developed a genetic approach with Saccharomyces cerevisiae. We constructed a yeast reporter strain containing the lacZ gene under the control of the CYC1 promoter associated with three copies of TRE-1. Expression of either the cyclic AMP response element-binding protein (CREB) or CREB fused to the GAL4 activation domain (GAD) in this strain did not modify the expression of the reporter gene. Tax alone was also inactive.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RlOxXPb1KMky"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.chemicals\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_chemicals_en.html)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 237
    },
    "id": "UdWgTerBKUfT",
    "outputId": "be43d334-3411-4ebd-d18f-f5d038904e65"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_chemicals download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-35c3fee2-18b5-43a5-8d37-a459526c6f12\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_chemicals</th>\n",
       "      <th>entities_chemicals_class</th>\n",
       "      <th>entities_chemicals_confidence</th>\n",
       "      <th>entities_chemicals_origin_chunk</th>\n",
       "      <th>entities_chemicals_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>cyclic AMP</td>\n",
       "      <td>CHEM</td>\n",
       "      <td>0.84725</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-35c3fee2-18b5-43a5-8d37-a459526c6f12')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-35c3fee2-18b5-43a5-8d37-a459526c6f12 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-35c3fee2-18b5-43a5-8d37-a459526c6f12');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_chemicals  \\\n",
       "0  Detection of various other intracellular signa...         cyclic AMP   \n",
       "\n",
       "  entities_chemicals_class entities_chemicals_confidence  \\\n",
       "0                     CHEM                       0.84725   \n",
       "\n",
       "  entities_chemicals_origin_chunk entities_chemicals_origin_sentence  \\\n",
       "0                               0                                  0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.chemicals').predict('Detection of various other intracellular signaling proteins is also described. Genetic characterization of transactivation of the human T-cell leukemia virus type 1 promoter: Binding of Tax to Tax-responsive element 1 is mediated by the cyclic AMP-responsive members of the CREB/ATF family of transcription factors. To achieve a better understanding of the mechanism of transactivation by Tax of human T-cell leukemia virus type 1 Tax-responsive element 1 (TRE-1), we developed a genetic approach with Saccharomyces cerevisiae. We constructed a yeast reporter strain containing the lacZ gene under the control of the CYC1 promoter associated with three copies of TRE-1. Expression of either the cyclic AMP response element-binding protein (CREB) or CREB fused to the GAL4 activation domain (GAD) in this strain did not modify the expression of the reporter gene. Tax alone was also inactive.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "lJ6DbD9lKqMU"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.chemprot\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_chemprot_biobert_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 529
    },
    "id": "A7Hxqua0LA2j",
    "outputId": "7b905d4a-6e42-4f31-c060-e695c564d466"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_chemprot_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-265f6536-8726-48fb-84da-0cac2ec04bdc\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_chemprot</th>\n",
       "      <th>entities_chemprot_class</th>\n",
       "      <th>entities_chemprot_confidence</th>\n",
       "      <th>entities_chemprot_origin_chunk</th>\n",
       "      <th>entities_chemprot_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>Keratinocyte</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.894</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.1026337519288063, -0.3668226897716522, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>growth</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.4833</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.1026337519288063, -0.3668226897716522, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>factor</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.7991</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.1026337519288063, -0.3668226897716522, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>acidic</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.9765</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.1026337519288063, -0.3668226897716522, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>fibroblast</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.3905</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.1026337519288063, -0.3668226897716522, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>growth</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.7109</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.1026337519288063, -0.3668226897716522, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>factor</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.8693</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.1026337519288063, -0.3668226897716522, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-265f6536-8726-48fb-84da-0cac2ec04bdc')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-265f6536-8726-48fb-84da-0cac2ec04bdc button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-265f6536-8726-48fb-84da-0cac2ec04bdc');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-2a007a0b-b0fb-4c1e-b88d-6d9fc699d9d2\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-2a007a0b-b0fb-4c1e-b88d-6d9fc699d9d2')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-2a007a0b-b0fb-4c1e-b88d-6d9fc699d9d2 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_chemprot  \\\n",
       "0  Keratinocyte growth factor and acidic fibrobla...      Keratinocyte   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            growth   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            factor   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            acidic   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...        fibroblast   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            growth   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            factor   \n",
       "\n",
       "  entities_chemprot_class entities_chemprot_confidence  \\\n",
       "0                  GENE-Y                        0.894   \n",
       "0                  GENE-Y                       0.4833   \n",
       "0                  GENE-Y                       0.7991   \n",
       "0                  GENE-Y                       0.9765   \n",
       "0                  GENE-Y                       0.3905   \n",
       "0                  GENE-Y                       0.7109   \n",
       "0                  GENE-Y                       0.8693   \n",
       "\n",
       "  entities_chemprot_origin_chunk entities_chemprot_origin_sentence  \\\n",
       "0                              0                                 0   \n",
       "0                              1                                 0   \n",
       "0                              2                                 0   \n",
       "0                              3                                 0   \n",
       "0                              4                                 0   \n",
       "0                              5                                 0   \n",
       "0                              6                                 0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-0.1026337519288063, -0.3668226897716522, 0....  \n",
       "0  [[-0.1026337519288063, -0.3668226897716522, 0....  \n",
       "0  [[-0.1026337519288063, -0.3668226897716522, 0....  \n",
       "0  [[-0.1026337519288063, -0.3668226897716522, 0....  \n",
       "0  [[-0.1026337519288063, -0.3668226897716522, 0....  \n",
       "0  [[-0.1026337519288063, -0.3668226897716522, 0....  \n",
       "0  [[-0.1026337519288063, -0.3668226897716522, 0....  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.chemprot').predict('Keratinocyte growth factor and acidic fibroblast growth factor are mitogens for primary cultures of mammary epithelium.', output_level='chunk')\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cLT9_iBNLcNj"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.chemprot.clinical\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_chemprot_clinical_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 529
    },
    "id": "eG4ecSecLlfM",
    "outputId": "0bb05cb1-f3ed-4ee9-a403-0864139d50e6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_chemprot_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-80f61e20-cb5b-43a9-b739-29033db36a9d\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_chemprot</th>\n",
       "      <th>entities_chemprot_class</th>\n",
       "      <th>entities_chemprot_confidence</th>\n",
       "      <th>entities_chemprot_origin_chunk</th>\n",
       "      <th>entities_chemprot_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>Keratinocyte</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.7433</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.22940552234649658, -0.21701078116893768, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>growth</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.6481</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.22940552234649658, -0.21701078116893768, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>factor</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.5693</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.22940552234649658, -0.21701078116893768, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>acidic</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.5518</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.22940552234649658, -0.21701078116893768, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>fibroblast</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.5111</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.22940552234649658, -0.21701078116893768, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>growth</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.4559</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.22940552234649658, -0.21701078116893768, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Keratinocyte growth factor and acidic fibrobla...</td>\n",
       "      <td>factor</td>\n",
       "      <td>GENE-Y</td>\n",
       "      <td>0.5213</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.22940552234649658, -0.21701078116893768, ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-80f61e20-cb5b-43a9-b739-29033db36a9d')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-80f61e20-cb5b-43a9-b739-29033db36a9d button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-80f61e20-cb5b-43a9-b739-29033db36a9d');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-eae08dda-2d40-424e-b251-11ba9f4c79f1\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-eae08dda-2d40-424e-b251-11ba9f4c79f1')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-eae08dda-2d40-424e-b251-11ba9f4c79f1 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_chemprot  \\\n",
       "0  Keratinocyte growth factor and acidic fibrobla...      Keratinocyte   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            growth   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            factor   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            acidic   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...        fibroblast   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            growth   \n",
       "0  Keratinocyte growth factor and acidic fibrobla...            factor   \n",
       "\n",
       "  entities_chemprot_class entities_chemprot_confidence  \\\n",
       "0                  GENE-Y                       0.7433   \n",
       "0                  GENE-Y                       0.6481   \n",
       "0                  GENE-Y                       0.5693   \n",
       "0                  GENE-Y                       0.5518   \n",
       "0                  GENE-Y                       0.5111   \n",
       "0                  GENE-Y                       0.4559   \n",
       "0                  GENE-Y                       0.5213   \n",
       "\n",
       "  entities_chemprot_origin_chunk entities_chemprot_origin_sentence  \\\n",
       "0                              0                                 0   \n",
       "0                              1                                 0   \n",
       "0                              2                                 0   \n",
       "0                              3                                 0   \n",
       "0                              4                                 0   \n",
       "0                              5                                 0   \n",
       "0                              6                                 0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.22940552234649658, -0.21701078116893768, ...  \n",
       "0  [[-0.22940552234649658, -0.21701078116893768, ...  \n",
       "0  [[-0.22940552234649658, -0.21701078116893768, ...  \n",
       "0  [[-0.22940552234649658, -0.21701078116893768, ...  \n",
       "0  [[-0.22940552234649658, -0.21701078116893768, ...  \n",
       "0  [[-0.22940552234649658, -0.21701078116893768, ...  \n",
       "0  [[-0.22940552234649658, -0.21701078116893768, ...  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.chemprot.clinical').predict('Keratinocyte growth factor and acidic fibroblast growth factor are mitogens for primary cultures of mammary epithelium.', output_level='chunk')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I2M-ZGLfMGCV"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.clinical\t\t\t](https://nlp.johnsnowlabs.com/2020/01/30/ner_clinical_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 286
    },
    "id": "rpKPlhPTMd0t",
    "outputId": "4758e745-b524-4753-8311-198fe9baa2db"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_chemprot_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-eb933b97-2cde-4fe8-b21b-567352bea60e\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_chemprot</th>\n",
       "      <th>entities_chemprot_class</th>\n",
       "      <th>entities_chemprot_confidence</th>\n",
       "      <th>entities_chemprot_origin_chunk</th>\n",
       "      <th>entities_chemprot_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Tylenol</td>\n",
       "      <td>CHEMICAL</td>\n",
       "      <td>0.9744</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>albuterol</td>\n",
       "      <td>CHEMICAL</td>\n",
       "      <td>0.9885</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-eb933b97-2cde-4fe8-b21b-567352bea60e')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-eb933b97-2cde-4fe8-b21b-567352bea60e button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-eb933b97-2cde-4fe8-b21b-567352bea60e');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-0eb534b6-d0cb-4a3c-8c36-78672e8384e7\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0eb534b6-d0cb-4a3c-8c36-78672e8384e7')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-0eb534b6-d0cb-4a3c-8c36-78672e8384e7 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_chemprot  \\\n",
       "0  The patient is a 21-day-old Caucasian male her...           Tylenol   \n",
       "0  The patient is a 21-day-old Caucasian male her...         albuterol   \n",
       "\n",
       "  entities_chemprot_class entities_chemprot_confidence  \\\n",
       "0                CHEMICAL                       0.9744   \n",
       "0                CHEMICAL                       0.9885   \n",
       "\n",
       "  entities_chemprot_origin_chunk entities_chemprot_origin_sentence  \\\n",
       "0                              0                                 0   \n",
       "0                              1                                 0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.chemprot.clinical').predict(\"The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature.\", output_level='chunk')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "T28KLQP2M6TU"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.clinical.biobert\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_clinical_biobert_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 627
    },
    "id": "i-flcQ8zNDHd",
    "outputId": "3baff2e2-d5a3-4652-bc13-a6bf1602da91"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_clinical_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-ce9fccc2-4829-4e5f-8727-8a817a555ca3\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_clinical</th>\n",
       "      <th>entities_clinical_class</th>\n",
       "      <th>entities_clinical_confidence</th>\n",
       "      <th>entities_clinical_origin_chunk</th>\n",
       "      <th>entities_clinical_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>congestion</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.9445</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>some mild problems with his breathing</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.7217</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>any perioral cyanosis</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.7104333</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>retractions</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.9969</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>a tactile temperature</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.8103333</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>some decreased p.o</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.776</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>his respiratory congestion</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.7612333</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>more tired</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.73155</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>fussy</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.8743</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ce9fccc2-4829-4e5f-8727-8a817a555ca3')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-ce9fccc2-4829-4e5f-8727-8a817a555ca3 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-ce9fccc2-4829-4e5f-8727-8a817a555ca3');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-0430af17-2e86-4e95-82dc-85667a92c97a\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0430af17-2e86-4e95-82dc-85667a92c97a')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-0430af17-2e86-4e95-82dc-85667a92c97a button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "\n",
       "                       entities_clinical entities_clinical_class  \\\n",
       "0                             congestion                 PROBLEM   \n",
       "0  some mild problems with his breathing                 PROBLEM   \n",
       "0                  any perioral cyanosis                 PROBLEM   \n",
       "0                            retractions                 PROBLEM   \n",
       "0                  a tactile temperature                 PROBLEM   \n",
       "0                     some decreased p.o                 PROBLEM   \n",
       "0             his respiratory congestion                 PROBLEM   \n",
       "0                             more tired                 PROBLEM   \n",
       "0                                  fussy                 PROBLEM   \n",
       "\n",
       "  entities_clinical_confidence entities_clinical_origin_chunk  \\\n",
       "0                       0.9445                              0   \n",
       "0                       0.7217                              1   \n",
       "0                    0.7104333                              2   \n",
       "0                       0.9969                              3   \n",
       "0                    0.8103333                              4   \n",
       "0                        0.776                              5   \n",
       "0                    0.7612333                              6   \n",
       "0                      0.73155                              7   \n",
       "0                       0.8743                              8   \n",
       "\n",
       "  entities_clinical_origin_sentence  \\\n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "0                                 0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.clinical.biobert').predict(\"The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature.\", output_level='chunk')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "p-rPcOCyNrvJ"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.diseases\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_diseases_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 286
    },
    "id": "gjl2JHIuN47-",
    "outputId": "f8385be2-51cc-4124-d3aa-3ba04053df32"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_diseases download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-8b28bb3b-ce04-4182-b0ce-7cc91b04cb93\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_diseases</th>\n",
       "      <th>entities_diseases_class</th>\n",
       "      <th>entities_diseases_confidence</th>\n",
       "      <th>entities_diseases_origin_chunk</th>\n",
       "      <th>entities_diseases_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>T-cell leukemia</td>\n",
       "      <td>Disease</td>\n",
       "      <td>0.7111</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>T-cell leukemia</td>\n",
       "      <td>Disease</td>\n",
       "      <td>0.8556</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.14205409586429596, 0.10900253802537918, 0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8b28bb3b-ce04-4182-b0ce-7cc91b04cb93')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-8b28bb3b-ce04-4182-b0ce-7cc91b04cb93 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-8b28bb3b-ce04-4182-b0ce-7cc91b04cb93');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-422869d2-c788-4020-a433-5b76b015992d\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-422869d2-c788-4020-a433-5b76b015992d')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-422869d2-c788-4020-a433-5b76b015992d button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_diseases  \\\n",
       "0  Detection of various other intracellular signa...   T-cell leukemia   \n",
       "0  Detection of various other intracellular signa...   T-cell leukemia   \n",
       "\n",
       "  entities_diseases_class entities_diseases_confidence  \\\n",
       "0                 Disease                       0.7111   \n",
       "0                 Disease                       0.8556   \n",
       "\n",
       "  entities_diseases_origin_chunk entities_diseases_origin_sentence  \\\n",
       "0                              0                                 0   \n",
       "0                              1                                 0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  \n",
       "0  [[-0.14205409586429596, 0.10900253802537918, 0...  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.diseases').predict(\"Detection of various other intracellular signaling proteins is also described. Genetic characterization of transactivation of the human T-cell leukemia virus type 1 promoter: Binding of Tax to Tax-responsive element 1 is mediated by the cyclic AMP-responsive members of the CREB/ATF family of transcription factors. To achieve a better understanding of the mechanism of transactivation by Tax of human T-cell leukemia virus type 1 Tax-responsive element 1 (TRE-1), we developed a genetic approach with Saccharomyces cerevisiae. We constructed a yeast reporter strain containing the lacZ gene under the control of the CYC1 promoter associated with three copies of TRE-1. Expression of either the cyclic AMP response element-binding protein (CREB) or CREB fused to the GAL4 activation domain (GAD) in this strain did not modify the expression of the reporter gene. Tax alone was also inactive. \", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yCUn4wR6OMun"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.diseases.biobert\t\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_diseases_biobert_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 286
    },
    "id": "a6STKFipOaZB",
    "outputId": "d407ea42-5674-48ee-b234-52443cdf2395"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_diseases_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-e9bf17e6-c87e-4345-9898-a7820cf5568d\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_diseases</th>\n",
       "      <th>entities_diseases_class</th>\n",
       "      <th>entities_diseases_confidence</th>\n",
       "      <th>entities_diseases_origin_chunk</th>\n",
       "      <th>entities_diseases_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>human T-cell leukemia</td>\n",
       "      <td>Disease</td>\n",
       "      <td>0.77633333</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Detection of various other intracellular signa...</td>\n",
       "      <td>T-cell leukemia</td>\n",
       "      <td>Disease</td>\n",
       "      <td>0.7939</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.03562016412615776, -0.31654998660087585, 0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e9bf17e6-c87e-4345-9898-a7820cf5568d')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-e9bf17e6-c87e-4345-9898-a7820cf5568d button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-e9bf17e6-c87e-4345-9898-a7820cf5568d');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-e965e9f5-730d-46e8-acc3-342bd4064cbb\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e965e9f5-730d-46e8-acc3-342bd4064cbb')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-e965e9f5-730d-46e8-acc3-342bd4064cbb button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document      entities_diseases  \\\n",
       "0  Detection of various other intracellular signa...  human T-cell leukemia   \n",
       "0  Detection of various other intracellular signa...        T-cell leukemia   \n",
       "\n",
       "  entities_diseases_class entities_diseases_confidence  \\\n",
       "0                 Disease                   0.77633333   \n",
       "0                 Disease                       0.7939   \n",
       "\n",
       "  entities_diseases_origin_chunk entities_diseases_origin_sentence  \\\n",
       "0                              0                                 0   \n",
       "0                              1                                 0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  \n",
       "0  [[0.03562016412615776, -0.31654998660087585, 0...  "
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.diseases.biobert').predict(\"Detection of various other intracellular signaling proteins is also described. Genetic characterization of transactivation of the human T-cell leukemia virus type 1 promoter: Binding of Tax to Tax-responsive element 1 is mediated by the cyclic AMP-responsive members of the CREB/ATF family of transcription factors. To achieve a better understanding of the mechanism of transactivation by Tax of human T-cell leukemia virus type 1 Tax-responsive element 1 (TRE-1), we developed a genetic approach with Saccharomyces cerevisiae. We constructed a yeast reporter strain containing the lacZ gene under the control of the CYC1 promoter associated with three copies of TRE-1. Expression of either the cyclic AMP response element-binding protein (CREB) or CREB fused to the GAL4 activation domain (GAD) in this strain did not modify the expression of the reporter gene. Tax alone was also inactive. \", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yURjTSZzOx84"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.drugs\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_drugs_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 529
    },
    "id": "qmxvoW0bO8W0",
    "outputId": "8c902a80-0be4-479d-e877-404643f1ee5e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_drugs download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-1cdf6bcd-90e4-4069-9649-4debb64dbf49\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_drugs</th>\n",
       "      <th>entities_drugs_class</th>\n",
       "      <th>entities_drugs_confidence</th>\n",
       "      <th>entities_drugs_origin_chunk</th>\n",
       "      <th>entities_drugs_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>potassium</td>\n",
       "      <td>DrugChem</td>\n",
       "      <td>0.5903</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>anthracyclines</td>\n",
       "      <td>DrugChem</td>\n",
       "      <td>0.9562</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>taxanes</td>\n",
       "      <td>DrugChem</td>\n",
       "      <td>0.5376</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>vinorelbine</td>\n",
       "      <td>DrugChem</td>\n",
       "      <td>0.9614</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>vinorelbine</td>\n",
       "      <td>DrugChem</td>\n",
       "      <td>0.9702</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>anthracyclines</td>\n",
       "      <td>DrugChem</td>\n",
       "      <td>0.9219</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>taxanes</td>\n",
       "      <td>DrugChem</td>\n",
       "      <td>0.5637</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1cdf6bcd-90e4-4069-9649-4debb64dbf49')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-1cdf6bcd-90e4-4069-9649-4debb64dbf49 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-1cdf6bcd-90e4-4069-9649-4debb64dbf49');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-aac12f1a-dc2c-4b79-a5c6-ce40876cff36\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-aac12f1a-dc2c-4b79-a5c6-ce40876cff36')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-aac12f1a-dc2c-4b79-a5c6-ce40876cff36 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  entities_drugs  \\\n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...       potassium   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...  anthracyclines   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...         taxanes   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...     vinorelbine   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...     vinorelbine   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...  anthracyclines   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...         taxanes   \n",
       "\n",
       "  entities_drugs_class entities_drugs_confidence entities_drugs_origin_chunk  \\\n",
       "0             DrugChem                    0.5903                           0   \n",
       "0             DrugChem                    0.9562                           1   \n",
       "0             DrugChem                    0.5376                           2   \n",
       "0             DrugChem                    0.9614                           3   \n",
       "0             DrugChem                    0.9702                           4   \n",
       "0             DrugChem                    0.9219                           5   \n",
       "0             DrugChem                    0.5637                           6   \n",
       "\n",
       "  entities_drugs_origin_sentence  \\\n",
       "0                              0   \n",
       "0                              0   \n",
       "0                              0   \n",
       "0                              0   \n",
       "0                              0   \n",
       "0                              0   \n",
       "0                              0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.drugs').predict(\"The human KCNJ9 (Kir 3.3, GIRK3) is a member of the G-protein-activated inwardly rectifying potassium (GIRK) channel family. Here we describe the genomicorganization of the KCNJ9 locus on chromosome 1q21-23 as a candidate gene forType II diabetes mellitus in the Pima Indian population. The gene spansapproximately 7.6 kb and contains one noncoding and two coding exons separated byapproximately 2.2 and approximately 2.6 kb introns, respectively. We identified14 single nucleotide polymorphisms (SNPs), including one that predicts aVal366Ala substitution, and an 8 base-pair (bp) insertion/deletion. Ourexpression studies revealed the presence of the transcript in various humantissues including pancreas, and two major insulin-responsive tissues: fat andskeletal muscle. The characterization of the KCNJ9 gene should facilitate furtherstudies on the function of the KCNJ9 protein and allow evaluation of thepotential role of the locus in Type II diabetes.BACKGROUND: At present, it is one of the most important issues for the treatment of breast cancer to develop the standard therapy for patients previously treated with anthracyclines and taxanes. With the objective of determining the usefulnessof vinorelbine monotherapy in patients with advanced or recurrent breast cancerafter standard therapy, we evaluated the efficacy and safety of vinorelbine inpatients previously treated with anthracyclines and taxanes.. \", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "x6DHVtxuPaC5"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.events_biobert\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_events_biobert_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 615
    },
    "id": "dwUrJQCLPkuE",
    "outputId": "ee981b26-b4b8-48fa-9a22-4ab14d2d598b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_events_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-2633fd7f-1f30-41b6-a0ea-798038e70f0e\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_events_biobert</th>\n",
       "      <th>entities_events_biobert_class</th>\n",
       "      <th>entities_events_biobert_confidence</th>\n",
       "      <th>entities_events_biobert_origin_chunk</th>\n",
       "      <th>entities_events_biobert_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>Kir</td>\n",
       "      <td>TEST</td>\n",
       "      <td>0.4799</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.6279052495956421, -0.10282155871391296, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>the G-protein-activated inwardly rectifying po...</td>\n",
       "      <td>TREATMENT</td>\n",
       "      <td>0.54476</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.6279052495956421, -0.10282155871391296, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The human KCNJ9 (Kir 3.3, GIRK3) is a member o...</td>\n",
       "      <td>diabetes mellitus</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.8075</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.6279052495956421, -0.10282155871391296, -...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-2633fd7f-1f30-41b6-a0ea-798038e70f0e')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-2633fd7f-1f30-41b6-a0ea-798038e70f0e button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-2633fd7f-1f30-41b6-a0ea-798038e70f0e');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-522e276e-b83a-45fe-86e7-c1f0e4769eec\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-522e276e-b83a-45fe-86e7-c1f0e4769eec')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-522e276e-b83a-45fe-86e7-c1f0e4769eec button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "0  The human KCNJ9 (Kir 3.3, GIRK3) is a member o...   \n",
       "\n",
       "                             entities_events_biobert  \\\n",
       "0                                                Kir   \n",
       "0  the G-protein-activated inwardly rectifying po...   \n",
       "0                                  diabetes mellitus   \n",
       "\n",
       "  entities_events_biobert_class entities_events_biobert_confidence  \\\n",
       "0                          TEST                             0.4799   \n",
       "0                     TREATMENT                            0.54476   \n",
       "0                       PROBLEM                             0.8075   \n",
       "\n",
       "  entities_events_biobert_origin_chunk  \\\n",
       "0                                    0   \n",
       "0                                    1   \n",
       "0                                    2   \n",
       "\n",
       "  entities_events_biobert_origin_sentence  \\\n",
       "0                                       0   \n",
       "0                                       0   \n",
       "0                                       0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-0.6279052495956421, -0.10282155871391296, -...  \n",
       "0  [[-0.6279052495956421, -0.10282155871391296, -...  \n",
       "0  [[-0.6279052495956421, -0.10282155871391296, -...  "
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.events_biobert').predict(\"The human KCNJ9 (Kir 3.3, GIRK3) is a member of the G-protein-activated inwardly rectifying potassium (GIRK) channel family. Here we describe the genomicorganization of the KCNJ9 locus on chromosome 1q21-23 as a candidate gene forType II diabetes mellitus in the Pima Indian population. The gene spansapproximately 7.6 kb and contains one noncoding and two coding exons separated byapproximately 2.2 and approximately 2.6 kb introns, respectively. We identified14 single nucleotide polymorphisms (SNPs), including one that predicts aVal366Ala substitution, and an 8 base-pair (bp) insertion/deletion. Ourexpression studies revealed the presence of the transcript in various humantissues including pancreas, and two major insulin-responsive tissues: fat andskeletal muscle. The characterization of the KCNJ9 gene should facilitate furtherstudies on the function of the KCNJ9 protein and allow evaluation of thepotential role of the locus in Type II diabetes.BACKGROUND: At present, it is one of the most important issues for the treatment of breast cancer to develop the standard therapy for patients previously treated with anthracyclines and taxanes. With the objective of determining the usefulnessof vinorelbine monotherapy in patients with advanced or recurrent breast cancerafter standard therapy, we evaluated the efficacy and safety of vinorelbine inpatients previously treated with anthracyclines and taxanes.. \", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MY5_9XNvPzt5"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.events_clinical\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_events_clinical_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 563
    },
    "id": "puyAAEPHQNWa",
    "outputId": "d58d9cc0-93af-47fb-e32c-ccb3cd0f4dc1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_events_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-ac8d6dac-c1da-48db-add7-02e5d1bdc3c0\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_events_clinical</th>\n",
       "      <th>entities_events_clinical_class</th>\n",
       "      <th>entities_events_clinical_confidence</th>\n",
       "      <th>entities_events_clinical_origin_chunk</th>\n",
       "      <th>entities_events_clinical_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient presented to the emergency room la...</td>\n",
       "      <td>presented</td>\n",
       "      <td>OCCURRENCE</td>\n",
       "      <td>0.6075</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient presented to the emergency room la...</td>\n",
       "      <td>the emergency room</td>\n",
       "      <td>CLINICAL_DEPT</td>\n",
       "      <td>0.8299667</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient presented to the emergency room la...</td>\n",
       "      <td>last evening</td>\n",
       "      <td>DATE</td>\n",
       "      <td>0.6577</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ac8d6dac-c1da-48db-add7-02e5d1bdc3c0')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-ac8d6dac-c1da-48db-add7-02e5d1bdc3c0 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-ac8d6dac-c1da-48db-add7-02e5d1bdc3c0');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-0b884a22-d054-436c-9d4e-beeac54061a7\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0b884a22-d054-436c-9d4e-beeac54061a7')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-0b884a22-d054-436c-9d4e-beeac54061a7 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_events_clinical  \\\n",
       "0  The patient presented to the emergency room la...                presented   \n",
       "0  The patient presented to the emergency room la...       the emergency room   \n",
       "0  The patient presented to the emergency room la...             last evening   \n",
       "\n",
       "  entities_events_clinical_class entities_events_clinical_confidence  \\\n",
       "0                     OCCURRENCE                              0.6075   \n",
       "0                  CLINICAL_DEPT                           0.8299667   \n",
       "0                           DATE                              0.6577   \n",
       "\n",
       "  entities_events_clinical_origin_chunk  \\\n",
       "0                                     0   \n",
       "0                                     1   \n",
       "0                                     2   \n",
       "\n",
       "  entities_events_clinical_origin_sentence  \\\n",
       "0                                        0   \n",
       "0                                        0   \n",
       "0                                        0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.events_clinical').predict(\"The patient presented to the emergency room last evening\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vsifIShMQlVA"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.events_healthcre\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_events_healthcare_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 563
    },
    "id": "C3LxHr0FQwLt",
    "outputId": "7e924768-ddc9-4d17-a52e-6c89ffee84fe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_events_healthcare download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_healthcare_100d download started this may take some time.\n",
      "Approximate size to download 475.8 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-77305d41-c6ab-44e6-add0-7aa9ad903448\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_events_healthcre</th>\n",
       "      <th>entities_events_healthcre_class</th>\n",
       "      <th>entities_events_healthcre_confidence</th>\n",
       "      <th>entities_events_healthcre_origin_chunk</th>\n",
       "      <th>entities_events_healthcre_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient presented to the emergency room la...</td>\n",
       "      <td>presented</td>\n",
       "      <td>EVIDENTIAL</td>\n",
       "      <td>0.6382</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient presented to the emergency room la...</td>\n",
       "      <td>the emergency room</td>\n",
       "      <td>CLINICAL_DEPT</td>\n",
       "      <td>0.8367667</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient presented to the emergency room la...</td>\n",
       "      <td>last evening</td>\n",
       "      <td>DATE</td>\n",
       "      <td>0.4767</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-77305d41-c6ab-44e6-add0-7aa9ad903448')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-77305d41-c6ab-44e6-add0-7aa9ad903448 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-77305d41-c6ab-44e6-add0-7aa9ad903448');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-e2141705-38d7-41c4-909e-c94d28592ded\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e2141705-38d7-41c4-909e-c94d28592ded')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-e2141705-38d7-41c4-909e-c94d28592ded button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The patient presented to the emergency room la...   \n",
       "0  The patient presented to the emergency room la...   \n",
       "0  The patient presented to the emergency room la...   \n",
       "\n",
       "  entities_events_healthcre entities_events_healthcre_class  \\\n",
       "0                 presented                      EVIDENTIAL   \n",
       "0        the emergency room                   CLINICAL_DEPT   \n",
       "0              last evening                            DATE   \n",
       "\n",
       "  entities_events_healthcre_confidence entities_events_healthcre_origin_chunk  \\\n",
       "0                               0.6382                                      0   \n",
       "0                            0.8367667                                      1   \n",
       "0                               0.4767                                      2   \n",
       "\n",
       "  entities_events_healthcre_origin_sentence  \\\n",
       "0                                         0   \n",
       "0                                         0   \n",
       "0                                         0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0  [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0  [[-0.11429792642593384, 0.18037879467010498, -...  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.events_healthcre').predict(\"The patient presented to the emergency room last evening\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TWDZNjsDRKpU"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.financial_contract\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_financial_contract_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 220
    },
    "id": "zwio0Tx9Q5Ot",
    "outputId": "b779ee37-6d05-4e84-ecf3-e0678ee25b97"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_financial_contract download started this may take some time.\n",
      "[OK!]\n",
      "glove_6B_300 download started this may take some time.\n",
      "Approximate size to download 426.2 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-e78b7776-f699-4212-a5c2-8b6e16c3bd9a\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_financial_contract</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient presented to the emergency room la...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[[0.046560000628232956, 0.21318000555038452, -...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e78b7776-f699-4212-a5c2-8b6e16c3bd9a')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-e78b7776-f699-4212-a5c2-8b6e16c3bd9a button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-e78b7776-f699-4212-a5c2-8b6e16c3bd9a');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The patient presented to the emergency room la...   \n",
       "\n",
       "  entities_financial_contract  \\\n",
       "0                         NaN   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.046560000628232956, 0.21318000555038452, -...  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.financial_contract').predict(\"The patient presented to the emergency room last evening\", output_level='chunk')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RlxTNwwERkkE"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.healthcare\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_healthcare_de.html)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 386
    },
    "id": "HVGn7IlTRuD-",
    "outputId": "a4577a34-43e3-48ed-d994-c242833e3a3b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_healthcare download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_healthcare_100d download started this may take some time.\n",
      "Approximate size to download 475.8 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-80d1f947-6271-460a-a5b7-89a20a5a7bb9\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_healthcare</th>\n",
       "      <th>entities_healthcare_class</th>\n",
       "      <th>entities_healthcare_confidence</th>\n",
       "      <th>entities_healthcare_origin_chunk</th>\n",
       "      <th>entities_healthcare_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Das Kleinzellige Bronchialkarzinom (Kleinzelli...</td>\n",
       "      <td>Bronchialkarzinom</td>\n",
       "      <td>TREATMENT</td>\n",
       "      <td>0.5038</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0757998377084732, 0.055228136479854584, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Das Kleinzellige Bronchialkarzinom (Kleinzelli...</td>\n",
       "      <td>SCLC</td>\n",
       "      <td>PROBLEM</td>\n",
       "      <td>0.9608</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0757998377084732, 0.055228136479854584, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Das Kleinzellige Bronchialkarzinom (Kleinzelli...</td>\n",
       "      <td>Karzinom</td>\n",
       "      <td>TREATMENT</td>\n",
       "      <td>0.8369</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0757998377084732, 0.055228136479854584, -...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-80d1f947-6271-460a-a5b7-89a20a5a7bb9')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-80d1f947-6271-460a-a5b7-89a20a5a7bb9 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-80d1f947-6271-460a-a5b7-89a20a5a7bb9');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-9e239e42-ddca-4e15-a89d-4a188cbbd45c\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-9e239e42-ddca-4e15-a89d-4a188cbbd45c')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-9e239e42-ddca-4e15-a89d-4a188cbbd45c button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_healthcare  \\\n",
       "0  Das Kleinzellige Bronchialkarzinom (Kleinzelli...   Bronchialkarzinom   \n",
       "0  Das Kleinzellige Bronchialkarzinom (Kleinzelli...                SCLC   \n",
       "0  Das Kleinzellige Bronchialkarzinom (Kleinzelli...            Karzinom   \n",
       "\n",
       "  entities_healthcare_class entities_healthcare_confidence  \\\n",
       "0                 TREATMENT                         0.5038   \n",
       "0                   PROBLEM                         0.9608   \n",
       "0                 TREATMENT                         0.8369   \n",
       "\n",
       "  entities_healthcare_origin_chunk entities_healthcare_origin_sentence  \\\n",
       "0                                0                                   0   \n",
       "0                                1                                   0   \n",
       "0                                2                                   0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0757998377084732, 0.055228136479854584, -...  \n",
       "0  [[-0.0757998377084732, 0.055228136479854584, -...  \n",
       "0  [[-0.0757998377084732, 0.055228136479854584, -...  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.healthcare').predict(\"Das Kleinzellige Bronchialkarzinom (Kleinzelliger Lungenkrebs, SCLC) ist ein hochmalignes bronchogenes Karzinom\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cIHXsw3ISjUe"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.human_phenotype.gene_biobert\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_human_phenotype_gene_biobert_en.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 799
    },
    "id": "DLwkEkztS25g",
    "outputId": "48d22e05-bfdc-4081-8ada-37333661c6b7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_human_phenotype_gene_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-11077fa3-0c95-4be1-b9d9-8d0472e77638\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_human_phenotype</th>\n",
       "      <th>entities_human_phenotype_class</th>\n",
       "      <th>entities_human_phenotype_confidence</th>\n",
       "      <th>entities_human_phenotype_origin_chunk</th>\n",
       "      <th>entities_human_phenotype_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Here we presented a case (BS type) of a 17 yea...</td>\n",
       "      <td>type</td>\n",
       "      <td>GENE</td>\n",
       "      <td>0.9977</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.0729779303073883, -0.04915425926446915, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Here we presented a case (BS type) of a 17 yea...</td>\n",
       "      <td>polyhydramnios</td>\n",
       "      <td>HP</td>\n",
       "      <td>0.9949</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.0729779303073883, -0.04915425926446915, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Here we presented a case (BS type) of a 17 yea...</td>\n",
       "      <td>polyuria</td>\n",
       "      <td>HP</td>\n",
       "      <td>0.9955</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.0729779303073883, -0.04915425926446915, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Here we presented a case (BS type) of a 17 yea...</td>\n",
       "      <td>nephrocalcinosis</td>\n",
       "      <td>HP</td>\n",
       "      <td>0.995</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.0729779303073883, -0.04915425926446915, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Here we presented a case (BS type) of a 17 yea...</td>\n",
       "      <td>hypokalemia</td>\n",
       "      <td>HP</td>\n",
       "      <td>0.9986</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.0729779303073883, -0.04915425926446915, -0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-11077fa3-0c95-4be1-b9d9-8d0472e77638')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-11077fa3-0c95-4be1-b9d9-8d0472e77638 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-11077fa3-0c95-4be1-b9d9-8d0472e77638');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-65fefc22-da93-4013-bafc-3f96d77c6148\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-65fefc22-da93-4013-bafc-3f96d77c6148')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-65fefc22-da93-4013-bafc-3f96d77c6148 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_human_phenotype  \\\n",
       "0  Here we presented a case (BS type) of a 17 yea...                     type   \n",
       "0  Here we presented a case (BS type) of a 17 yea...           polyhydramnios   \n",
       "0  Here we presented a case (BS type) of a 17 yea...                 polyuria   \n",
       "0  Here we presented a case (BS type) of a 17 yea...         nephrocalcinosis   \n",
       "0  Here we presented a case (BS type) of a 17 yea...              hypokalemia   \n",
       "\n",
       "  entities_human_phenotype_class entities_human_phenotype_confidence  \\\n",
       "0                           GENE                              0.9977   \n",
       "0                             HP                              0.9949   \n",
       "0                             HP                              0.9955   \n",
       "0                             HP                               0.995   \n",
       "0                             HP                              0.9986   \n",
       "\n",
       "  entities_human_phenotype_origin_chunk  \\\n",
       "0                                     0   \n",
       "0                                     1   \n",
       "0                                     2   \n",
       "0                                     3   \n",
       "0                                     4   \n",
       "\n",
       "  entities_human_phenotype_origin_sentence  \\\n",
       "0                                        0   \n",
       "0                                        0   \n",
       "0                                        0   \n",
       "0                                        0   \n",
       "0                                        0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[0.0729779303073883, -0.04915425926446915, -0...  \n",
       "0  [[0.0729779303073883, -0.04915425926446915, -0...  \n",
       "0  [[0.0729779303073883, -0.04915425926446915, -0...  \n",
       "0  [[0.0729779303073883, -0.04915425926446915, -0...  \n",
       "0  [[0.0729779303073883, -0.04915425926446915, -0...  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.human_phenotype.gene_biobert').predict(\"Here we presented a case (BS type) of a 17 years old female presented with polyhydramnios, polyuria, nephrocalcinosis and hypokalemia, which was alleviated after treatment with celecoxib and vitamin D(3).\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8_FubHYKUNkd"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.human_phenotype.go_biobert\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_human_phenotype_go_biobert_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 681
    },
    "id": "IGBVhF2sUgmf",
    "outputId": "d25566f6-4c1d-478d-cbad-c7449098ebeb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_human_phenotype_go_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-ad54cb5a-2e0b-4670-99ab-177ebca2775a\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_human_phenotype</th>\n",
       "      <th>entities_human_phenotype_class</th>\n",
       "      <th>entities_human_phenotype_confidence</th>\n",
       "      <th>entities_human_phenotype_origin_chunk</th>\n",
       "      <th>entities_human_phenotype_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Here we presented a case (BS type) of a 17 yea...</td>\n",
       "      <td>polyhydramnios</td>\n",
       "      <td>HP</td>\n",
       "      <td>0.9963</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.0729779303073883, -0.04915425926446915, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Here we presented a case (BS type) of a 17 yea...</td>\n",
       "      <td>polyuria</td>\n",
       "      <td>HP</td>\n",
       "      <td>0.9998</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.0729779303073883, -0.04915425926446915, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Here we presented a case (BS type) of a 17 yea...</td>\n",
       "      <td>nephrocalcinosis</td>\n",
       "      <td>HP</td>\n",
       "      <td>0.9996</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.0729779303073883, -0.04915425926446915, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Here we presented a case (BS type) of a 17 yea...</td>\n",
       "      <td>hypokalemia</td>\n",
       "      <td>HP</td>\n",
       "      <td>0.9988</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.0729779303073883, -0.04915425926446915, -0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ad54cb5a-2e0b-4670-99ab-177ebca2775a')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-ad54cb5a-2e0b-4670-99ab-177ebca2775a button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-ad54cb5a-2e0b-4670-99ab-177ebca2775a');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-80d45b2c-007d-4cdc-b7bb-8642bef6a788\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-80d45b2c-007d-4cdc-b7bb-8642bef6a788')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-80d45b2c-007d-4cdc-b7bb-8642bef6a788 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_human_phenotype  \\\n",
       "0  Here we presented a case (BS type) of a 17 yea...           polyhydramnios   \n",
       "0  Here we presented a case (BS type) of a 17 yea...                 polyuria   \n",
       "0  Here we presented a case (BS type) of a 17 yea...         nephrocalcinosis   \n",
       "0  Here we presented a case (BS type) of a 17 yea...              hypokalemia   \n",
       "\n",
       "  entities_human_phenotype_class entities_human_phenotype_confidence  \\\n",
       "0                             HP                              0.9963   \n",
       "0                             HP                              0.9998   \n",
       "0                             HP                              0.9996   \n",
       "0                             HP                              0.9988   \n",
       "\n",
       "  entities_human_phenotype_origin_chunk  \\\n",
       "0                                     0   \n",
       "0                                     1   \n",
       "0                                     2   \n",
       "0                                     3   \n",
       "\n",
       "  entities_human_phenotype_origin_sentence  \\\n",
       "0                                        0   \n",
       "0                                        0   \n",
       "0                                        0   \n",
       "0                                        0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[0.0729779303073883, -0.04915425926446915, -0...  \n",
       "0  [[0.0729779303073883, -0.04915425926446915, -0...  \n",
       "0  [[0.0729779303073883, -0.04915425926446915, -0...  \n",
       "0  [[0.0729779303073883, -0.04915425926446915, -0...  "
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.human_phenotype.go_biobert').predict(\"Here we presented a case (BS type) of a 17 years old female presented with polyhydramnios, polyuria, nephrocalcinosis and hypokalemia, which was alleviated after treatment with celecoxib and vitamin D(3).\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1_qBMEn0URY2"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.jsl\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_jsl_en.html)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "xNR2MUxmVMPx",
    "outputId": "65b03fc8-77ef-46a2-f3e8-414d42d982de"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_jsl download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-ed8960b6-e217-4487-98a9-e4dc3f950d5b\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_jsl</th>\n",
       "      <th>entities_jsl_class</th>\n",
       "      <th>entities_jsl_confidence</th>\n",
       "      <th>entities_jsl_origin_chunk</th>\n",
       "      <th>entities_jsl_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>21-day-old</td>\n",
       "      <td>Age</td>\n",
       "      <td>0.9961</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>Race_Ethnicity</td>\n",
       "      <td>0.9994</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>male</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9998</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>2 days</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.7975</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8992</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.989</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>suctioning yellow discharge</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.2556</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>nares</td>\n",
       "      <td>External_body_part_or_region</td>\n",
       "      <td>0.7476</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>she</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9978</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mild</td>\n",
       "      <td>Modifier</td>\n",
       "      <td>0.6977</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>problems with his breathing while feeding</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.6648666</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>perioral cyanosis</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.7007</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>retractions</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9826</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>One day ago</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.74329996</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.999</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tactile temperature</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.29775</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Tylenol</td>\n",
       "      <td>Drug_BrandName</td>\n",
       "      <td>0.9963</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>decreased p.o</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.65505004</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9672</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>20 minutes</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.4626</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>5 to 10 minutes</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.53627497</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9719</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>respiratory congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.59889996</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9941</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tired</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8456</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>fussy</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8113</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>over the past 2 days</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.6722</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>albuterol</td>\n",
       "      <td>Drug_Ingredient</td>\n",
       "      <td>0.9821</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>ER</td>\n",
       "      <td>Clinical_Dept</td>\n",
       "      <td>0.9972</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9974</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>urine output has also decreased</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.42608</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9953</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>per 24 hours</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.4757</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9971</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>per 24 hours</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.47673333</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.997</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>diarrhea</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9855</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9901</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>bowel</td>\n",
       "      <td>Internal_organ_or_component</td>\n",
       "      <td>0.8241</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ed8960b6-e217-4487-98a9-e4dc3f950d5b')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-ed8960b6-e217-4487-98a9-e4dc3f950d5b button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-ed8960b6-e217-4487-98a9-e4dc3f950d5b');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-0ac08d98-db40-4ace-bb09-9904ba6fbce8\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-0ac08d98-db40-4ace-bb09-9904ba6fbce8')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-0ac08d98-db40-4ace-bb09-9904ba6fbce8 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "\n",
       "                                entities_jsl            entities_jsl_class  \\\n",
       "0                                 21-day-old                           Age   \n",
       "0                                  Caucasian                Race_Ethnicity   \n",
       "0                                       male                        Gender   \n",
       "0                                     2 days                      Duration   \n",
       "0                                 congestion                       Symptom   \n",
       "0                                        mom                        Gender   \n",
       "0                suctioning yellow discharge                       Symptom   \n",
       "0                                      nares  External_body_part_or_region   \n",
       "0                                        she                        Gender   \n",
       "0                                       mild                      Modifier   \n",
       "0  problems with his breathing while feeding                       Symptom   \n",
       "0                          perioral cyanosis                       Symptom   \n",
       "0                                retractions                       Symptom   \n",
       "0                                One day ago                  RelativeDate   \n",
       "0                                        mom                        Gender   \n",
       "0                        tactile temperature                       Symptom   \n",
       "0                                    Tylenol                Drug_BrandName   \n",
       "0                              decreased p.o                       Symptom   \n",
       "0                                        His                        Gender   \n",
       "0                                 20 minutes                      Duration   \n",
       "0                            5 to 10 minutes                      Duration   \n",
       "0                                        his                        Gender   \n",
       "0                     respiratory congestion                       Symptom   \n",
       "0                                         He                        Gender   \n",
       "0                                      tired                       Symptom   \n",
       "0                                      fussy                       Symptom   \n",
       "0                       over the past 2 days                  RelativeDate   \n",
       "0                                  albuterol               Drug_Ingredient   \n",
       "0                                         ER                 Clinical_Dept   \n",
       "0                                        His                        Gender   \n",
       "0            urine output has also decreased                       Symptom   \n",
       "0                                         he                        Gender   \n",
       "0                               per 24 hours                     Frequency   \n",
       "0                                         he                        Gender   \n",
       "0                               per 24 hours                     Frequency   \n",
       "0                                        Mom                        Gender   \n",
       "0                                   diarrhea                       Symptom   \n",
       "0                                        His                        Gender   \n",
       "0                                      bowel   Internal_organ_or_component   \n",
       "\n",
       "  entities_jsl_confidence entities_jsl_origin_chunk  \\\n",
       "0                  0.9961                         0   \n",
       "0                  0.9994                         1   \n",
       "0                  0.9998                         2   \n",
       "0                  0.7975                         3   \n",
       "0                  0.8992                         4   \n",
       "0                   0.989                         5   \n",
       "0                  0.2556                         6   \n",
       "0                  0.7476                         7   \n",
       "0                  0.9978                         8   \n",
       "0                  0.6977                         9   \n",
       "0               0.6648666                        10   \n",
       "0                  0.7007                        11   \n",
       "0                  0.9826                        12   \n",
       "0              0.74329996                        13   \n",
       "0                   0.999                        14   \n",
       "0                 0.29775                        15   \n",
       "0                  0.9963                        16   \n",
       "0              0.65505004                        17   \n",
       "0                  0.9672                        18   \n",
       "0                  0.4626                        19   \n",
       "0              0.53627497                        20   \n",
       "0                  0.9719                        21   \n",
       "0              0.59889996                        22   \n",
       "0                  0.9941                        23   \n",
       "0                  0.8456                        24   \n",
       "0                  0.8113                        25   \n",
       "0                  0.6722                        26   \n",
       "0                  0.9821                        27   \n",
       "0                  0.9972                        28   \n",
       "0                  0.9974                        29   \n",
       "0                 0.42608                        30   \n",
       "0                  0.9953                        31   \n",
       "0                  0.4757                        32   \n",
       "0                  0.9971                        33   \n",
       "0              0.47673333                        34   \n",
       "0                   0.997                        35   \n",
       "0                  0.9855                        36   \n",
       "0                  0.9901                        37   \n",
       "0                  0.8241                        38   \n",
       "\n",
       "  entities_jsl_origin_sentence  \\\n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.jsl').predict(\"The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature..\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xnUpJaOZVK2l"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.jsl.biobert\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_jsl_biobert_en.html)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "WeQI86AaVLfb",
    "outputId": "bffc49d4-93a9-4086-8cb1-d02e9bbfd72b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_jsl_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-4c6ca80b-9595-4692-902c-ca950752da5b\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_jsl</th>\n",
       "      <th>entities_jsl_class</th>\n",
       "      <th>entities_jsl_confidence</th>\n",
       "      <th>entities_jsl_origin_chunk</th>\n",
       "      <th>entities_jsl_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>21-day-old</td>\n",
       "      <td>Age</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>Race_Ethnicity</td>\n",
       "      <td>0.8431</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>male</td>\n",
       "      <td>Gender</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>for 2 days</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.50673336</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9288</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9995</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>suctioning yellow discharge</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.38756666</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>nares</td>\n",
       "      <td>External_body_part_or_region</td>\n",
       "      <td>0.9072</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>she</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9863</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mild</td>\n",
       "      <td>Modifier</td>\n",
       "      <td>0.5171</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>problems with his breathing while feeding</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.46468332</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>perioral cyanosis</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.60385</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>retractions</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.968</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>One day ago</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.62763333</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9974</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tactile temperature</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.58455</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Tylenol</td>\n",
       "      <td>Drug_BrandName</td>\n",
       "      <td>0.9987</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>decreased p.o</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.33505</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9885</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>from 20 minutes q.2h. to 5 to 10 minutes</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.26447</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9716</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>respiratory congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.53075</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9958</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tired</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8774</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>fussy</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.989</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>over the past</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.3317</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.8513102531433105, -0.7403942346572876, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4c6ca80b-9595-4692-902c-ca950752da5b')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-4c6ca80b-9595-4692-902c-ca950752da5b button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-4c6ca80b-9595-4692-902c-ca950752da5b');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-5f58e851-289c-4f68-bf52-eb90a648375c\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-5f58e851-289c-4f68-bf52-eb90a648375c')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-5f58e851-289c-4f68-bf52-eb90a648375c button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "\n",
       "                                entities_jsl            entities_jsl_class  \\\n",
       "0                                 21-day-old                           Age   \n",
       "0                                  Caucasian                Race_Ethnicity   \n",
       "0                                       male                        Gender   \n",
       "0                                 for 2 days                      Duration   \n",
       "0                                 congestion                       Symptom   \n",
       "0                                        mom                        Gender   \n",
       "0                suctioning yellow discharge                       Symptom   \n",
       "0                                      nares  External_body_part_or_region   \n",
       "0                                        she                        Gender   \n",
       "0                                       mild                      Modifier   \n",
       "0  problems with his breathing while feeding                       Symptom   \n",
       "0                          perioral cyanosis                       Symptom   \n",
       "0                                retractions                       Symptom   \n",
       "0                                One day ago                  RelativeDate   \n",
       "0                                        mom                        Gender   \n",
       "0                        tactile temperature                       Symptom   \n",
       "0                                    Tylenol                Drug_BrandName   \n",
       "0                              decreased p.o                       Symptom   \n",
       "0                                        His                        Gender   \n",
       "0   from 20 minutes q.2h. to 5 to 10 minutes                     Frequency   \n",
       "0                                        his                        Gender   \n",
       "0                     respiratory congestion                       Symptom   \n",
       "0                                         He                        Gender   \n",
       "0                                      tired                       Symptom   \n",
       "0                                      fussy                       Symptom   \n",
       "0                              over the past                  RelativeDate   \n",
       "\n",
       "  entities_jsl_confidence entities_jsl_origin_chunk  \\\n",
       "0                     1.0                         0   \n",
       "0                  0.8431                         1   \n",
       "0                     1.0                         2   \n",
       "0              0.50673336                         3   \n",
       "0                  0.9288                         4   \n",
       "0                  0.9995                         5   \n",
       "0              0.38756666                         6   \n",
       "0                  0.9072                         7   \n",
       "0                  0.9863                         8   \n",
       "0                  0.5171                         9   \n",
       "0              0.46468332                        10   \n",
       "0                 0.60385                        11   \n",
       "0                   0.968                        12   \n",
       "0              0.62763333                        13   \n",
       "0                  0.9974                        14   \n",
       "0                 0.58455                        15   \n",
       "0                  0.9987                        16   \n",
       "0                 0.33505                        17   \n",
       "0                  0.9885                        18   \n",
       "0                 0.26447                        19   \n",
       "0                  0.9716                        20   \n",
       "0                 0.53075                        21   \n",
       "0                  0.9958                        22   \n",
       "0                  0.8774                        23   \n",
       "0                   0.989                        24   \n",
       "0                  0.3317                        25   \n",
       "\n",
       "  entities_jsl_origin_sentence  \\\n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  \n",
       "0  [[-0.8513102531433105, -0.7403942346572876, 0....  "
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.jsl.biobert').predict(\"The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature..\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FA-9rBrnV5eS"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.jsl.enriched\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_jsl_enriched_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "9JdoG_LeWDbM",
    "outputId": "73e908a1-78e0-4554-c5e4-1cd04627f5b1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_jsl_enriched download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-ecdbacf7-09cf-447b-a173-630ac092f227\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_jsl</th>\n",
       "      <th>entities_jsl_class</th>\n",
       "      <th>entities_jsl_confidence</th>\n",
       "      <th>entities_jsl_origin_chunk</th>\n",
       "      <th>entities_jsl_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>21-day-old</td>\n",
       "      <td>Age</td>\n",
       "      <td>0.9994</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>Race_Ethnicity</td>\n",
       "      <td>0.9989</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>male</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9989</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>2 days</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.8464</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9919</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9835</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>suctioning yellow discharge</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.35636666</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>nares</td>\n",
       "      <td>External_body_part_or_region</td>\n",
       "      <td>0.861</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>she</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9974</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mild</td>\n",
       "      <td>Modifier</td>\n",
       "      <td>0.6101</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>problems with his breathing while feeding</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.60978335</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>perioral cyanosis</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.46755</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>retractions</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9965</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>One day ago</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.5448667</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9929</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tactile temperature</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.37325</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Tylenol</td>\n",
       "      <td>Drug_BrandName</td>\n",
       "      <td>0.9603</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Baby</td>\n",
       "      <td>Age</td>\n",
       "      <td>0.4651</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>decreased p.o. intake</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.560825</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.967</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>20 minutes q.2h. to 5 to 10 minutes</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.42732224</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.965</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>respiratory congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.5112</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9977</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tired</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9631</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>fussy</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.7463</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>over the past 2 days</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.59458005</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>albuterol</td>\n",
       "      <td>Drug_Ingredient</td>\n",
       "      <td>0.9897</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>ER</td>\n",
       "      <td>Clinical_Dept</td>\n",
       "      <td>0.9971</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9975</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>urine output has also decreased</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.22683999</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9829</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>per 24 hours</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.4844</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.993</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>per 24 hours</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.4373</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9793</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>diarrhea</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9985</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9839</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>bowel</td>\n",
       "      <td>Internal_organ_or_component</td>\n",
       "      <td>0.7868</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ecdbacf7-09cf-447b-a173-630ac092f227')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-ecdbacf7-09cf-447b-a173-630ac092f227 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-ecdbacf7-09cf-447b-a173-630ac092f227');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-c03c94b3-f49d-47e0-9478-075b7e2d3927\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c03c94b3-f49d-47e0-9478-075b7e2d3927')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-c03c94b3-f49d-47e0-9478-075b7e2d3927 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "\n",
       "                                entities_jsl            entities_jsl_class  \\\n",
       "0                                 21-day-old                           Age   \n",
       "0                                  Caucasian                Race_Ethnicity   \n",
       "0                                       male                        Gender   \n",
       "0                                     2 days                      Duration   \n",
       "0                                 congestion                       Symptom   \n",
       "0                                        mom                        Gender   \n",
       "0                suctioning yellow discharge                       Symptom   \n",
       "0                                      nares  External_body_part_or_region   \n",
       "0                                        she                        Gender   \n",
       "0                                       mild                      Modifier   \n",
       "0  problems with his breathing while feeding                       Symptom   \n",
       "0                          perioral cyanosis                       Symptom   \n",
       "0                                retractions                       Symptom   \n",
       "0                                One day ago                  RelativeDate   \n",
       "0                                        mom                        Gender   \n",
       "0                        tactile temperature                       Symptom   \n",
       "0                                    Tylenol                Drug_BrandName   \n",
       "0                                       Baby                           Age   \n",
       "0                      decreased p.o. intake                       Symptom   \n",
       "0                                        His                        Gender   \n",
       "0        20 minutes q.2h. to 5 to 10 minutes                      Duration   \n",
       "0                                        his                        Gender   \n",
       "0                     respiratory congestion                       Symptom   \n",
       "0                                         He                        Gender   \n",
       "0                                      tired                       Symptom   \n",
       "0                                      fussy                       Symptom   \n",
       "0                       over the past 2 days                  RelativeDate   \n",
       "0                                  albuterol               Drug_Ingredient   \n",
       "0                                         ER                 Clinical_Dept   \n",
       "0                                        His                        Gender   \n",
       "0            urine output has also decreased                       Symptom   \n",
       "0                                         he                        Gender   \n",
       "0                               per 24 hours                     Frequency   \n",
       "0                                         he                        Gender   \n",
       "0                               per 24 hours                     Frequency   \n",
       "0                                        Mom                        Gender   \n",
       "0                                   diarrhea                       Symptom   \n",
       "0                                        His                        Gender   \n",
       "0                                      bowel   Internal_organ_or_component   \n",
       "\n",
       "  entities_jsl_confidence entities_jsl_origin_chunk  \\\n",
       "0                  0.9994                         0   \n",
       "0                  0.9989                         1   \n",
       "0                  0.9989                         2   \n",
       "0                  0.8464                         3   \n",
       "0                  0.9919                         4   \n",
       "0                  0.9835                         5   \n",
       "0              0.35636666                         6   \n",
       "0                   0.861                         7   \n",
       "0                  0.9974                         8   \n",
       "0                  0.6101                         9   \n",
       "0              0.60978335                        10   \n",
       "0                 0.46755                        11   \n",
       "0                  0.9965                        12   \n",
       "0               0.5448667                        13   \n",
       "0                  0.9929                        14   \n",
       "0                 0.37325                        15   \n",
       "0                  0.9603                        16   \n",
       "0                  0.4651                        17   \n",
       "0                0.560825                        18   \n",
       "0                   0.967                        19   \n",
       "0              0.42732224                        20   \n",
       "0                   0.965                        21   \n",
       "0                  0.5112                        22   \n",
       "0                  0.9977                        23   \n",
       "0                  0.9631                        24   \n",
       "0                  0.7463                        25   \n",
       "0              0.59458005                        26   \n",
       "0                  0.9897                        27   \n",
       "0                  0.9971                        28   \n",
       "0                  0.9975                        29   \n",
       "0              0.22683999                        30   \n",
       "0                  0.9829                        31   \n",
       "0                  0.4844                        32   \n",
       "0                   0.993                        33   \n",
       "0                  0.4373                        34   \n",
       "0                  0.9793                        35   \n",
       "0                  0.9985                        36   \n",
       "0                  0.9839                        37   \n",
       "0                  0.7868                        38   \n",
       "\n",
       "  entities_jsl_origin_sentence  \\\n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.jsl.enriched').predict(\"The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature..\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yBs2vEmIWr-W"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.measurements\t\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_measurements_clinical_en.html)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 220
    },
    "id": "GIVYo7WLW1UZ",
    "outputId": "ed2ced03-b343-4757-908b-166b5ff89d1c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_measurements_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-916cf01b-0cd1-4825-a94f-89987070b9d9\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_measurements</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-916cf01b-0cd1-4825-a94f-89987070b9d9')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-916cf01b-0cd1-4825-a94f-89987070b9d9 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-916cf01b-0cd1-4825-a94f-89987070b9d9');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_measurements  \\\n",
       "0  The patient is a 21-day-old Caucasian male her...                   NaN   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.measurements').predict(\"The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature..\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tpqjQ9ySXM8C"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.medmentions\t\t\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_medmentions_coarse_en.html)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "vPLKWBB8W7Yk",
    "outputId": "8deecf1a-39e7-4505-de40-bcb64572620d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_medmentions_coarse download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-74a19e64-b069-4c26-88e5-59b4efda56f6\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_medmentions</th>\n",
       "      <th>entities_medmentions_class</th>\n",
       "      <th>entities_medmentions_confidence</th>\n",
       "      <th>entities_medmentions_origin_chunk</th>\n",
       "      <th>entities_medmentions_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>Population_Group</td>\n",
       "      <td>0.8195</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>male</td>\n",
       "      <td>Organism_Attribute</td>\n",
       "      <td>0.6742</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>congestion</td>\n",
       "      <td>Pathologic_Function</td>\n",
       "      <td>0.3855</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>suctioning yellow discharge</td>\n",
       "      <td>Therapeutic_or_Preventive_Procedure</td>\n",
       "      <td>0.28693333</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>nares</td>\n",
       "      <td>Body_Part,_Organ,_or_Organ_Component</td>\n",
       "      <td>0.5185</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>breathing</td>\n",
       "      <td>Biologic_Function</td>\n",
       "      <td>0.7695</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>perioral cyanosis</td>\n",
       "      <td>Sign_or_Symptom</td>\n",
       "      <td>0.49440002</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>temperature</td>\n",
       "      <td>Quantitative_Concept</td>\n",
       "      <td>0.378</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Tylenol</td>\n",
       "      <td>Organic_Chemical</td>\n",
       "      <td>0.6089</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>decreased</td>\n",
       "      <td>Quantitative_Concept</td>\n",
       "      <td>0.97</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>respiratory congestion</td>\n",
       "      <td>Pathologic_Function</td>\n",
       "      <td>0.44134998</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>albuterol</td>\n",
       "      <td>Organic_Chemical</td>\n",
       "      <td>0.2881</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>treatments</td>\n",
       "      <td>Therapeutic_or_Preventive_Procedure</td>\n",
       "      <td>0.3496</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>ER</td>\n",
       "      <td>Cell_Component</td>\n",
       "      <td>0.3037</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>urine</td>\n",
       "      <td>Body_Substance</td>\n",
       "      <td>0.9373</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>decreased</td>\n",
       "      <td>Quantitative_Concept</td>\n",
       "      <td>0.9571</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>diapers</td>\n",
       "      <td>Manufactured_Object</td>\n",
       "      <td>0.2128</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>diapers</td>\n",
       "      <td>Biomedical_or_Dental_Material</td>\n",
       "      <td>0.2113</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>diarrhea</td>\n",
       "      <td>Sign_or_Symptom</td>\n",
       "      <td>0.8604</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>bowel movements</td>\n",
       "      <td>Biologic_Function</td>\n",
       "      <td>0.36345</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>yellow</td>\n",
       "      <td>Qualitative_Concept</td>\n",
       "      <td>0.6888</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>colored</td>\n",
       "      <td>Qualitative_Concept</td>\n",
       "      <td>0.1926</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>soft</td>\n",
       "      <td>Qualitative_Concept</td>\n",
       "      <td>0.3926</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-74a19e64-b069-4c26-88e5-59b4efda56f6')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-74a19e64-b069-4c26-88e5-59b4efda56f6 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-74a19e64-b069-4c26-88e5-59b4efda56f6');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-7100a3ea-3dfd-4b51-b956-68c86321972a\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-7100a3ea-3dfd-4b51-b956-68c86321972a')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-7100a3ea-3dfd-4b51-b956-68c86321972a button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "\n",
       "          entities_medmentions            entities_medmentions_class  \\\n",
       "0                    Caucasian                      Population_Group   \n",
       "0                         male                    Organism_Attribute   \n",
       "0                   congestion                   Pathologic_Function   \n",
       "0  suctioning yellow discharge   Therapeutic_or_Preventive_Procedure   \n",
       "0                        nares  Body_Part,_Organ,_or_Organ_Component   \n",
       "0                    breathing                     Biologic_Function   \n",
       "0            perioral cyanosis                       Sign_or_Symptom   \n",
       "0                  temperature                  Quantitative_Concept   \n",
       "0                      Tylenol                      Organic_Chemical   \n",
       "0                    decreased                  Quantitative_Concept   \n",
       "0       respiratory congestion                   Pathologic_Function   \n",
       "0                    albuterol                      Organic_Chemical   \n",
       "0                   treatments   Therapeutic_or_Preventive_Procedure   \n",
       "0                           ER                        Cell_Component   \n",
       "0                        urine                        Body_Substance   \n",
       "0                    decreased                  Quantitative_Concept   \n",
       "0                      diapers                   Manufactured_Object   \n",
       "0                      diapers         Biomedical_or_Dental_Material   \n",
       "0                     diarrhea                       Sign_or_Symptom   \n",
       "0              bowel movements                     Biologic_Function   \n",
       "0                       yellow                   Qualitative_Concept   \n",
       "0                      colored                   Qualitative_Concept   \n",
       "0                         soft                   Qualitative_Concept   \n",
       "\n",
       "  entities_medmentions_confidence entities_medmentions_origin_chunk  \\\n",
       "0                          0.8195                                 0   \n",
       "0                          0.6742                                 1   \n",
       "0                          0.3855                                 2   \n",
       "0                      0.28693333                                 3   \n",
       "0                          0.5185                                 4   \n",
       "0                          0.7695                                 5   \n",
       "0                      0.49440002                                 6   \n",
       "0                           0.378                                 7   \n",
       "0                          0.6089                                 8   \n",
       "0                            0.97                                 9   \n",
       "0                      0.44134998                                10   \n",
       "0                          0.2881                                11   \n",
       "0                          0.3496                                12   \n",
       "0                          0.3037                                13   \n",
       "0                          0.9373                                14   \n",
       "0                          0.9571                                15   \n",
       "0                          0.2128                                16   \n",
       "0                          0.2113                                17   \n",
       "0                          0.8604                                18   \n",
       "0                         0.36345                                19   \n",
       "0                          0.6888                                20   \n",
       "0                          0.1926                                21   \n",
       "0                          0.3926                                22   \n",
       "\n",
       "  entities_medmentions_origin_sentence  \\\n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "0                                    0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.medmentions').predict(\"The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature..\", output_level='chunk')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XAqvdI25Xt5U"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.posology](https://nlp.johnsnowlabs.com/2020/04/15/ner_posology_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 736
    },
    "id": "tIBsi57jXzlx",
    "outputId": "c7ffb9fa-d6a7-4049-e953-cde4fa10f16a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_posology download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-1cdfd71e-c443-4eff-8438-e057176be8aa\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_posology</th>\n",
       "      <th>entities_posology_class</th>\n",
       "      <th>entities_posology_confidence</th>\n",
       "      <th>entities_posology_origin_chunk</th>\n",
       "      <th>entities_posology_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>insulin</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.9858</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Bactrim</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.9964</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>for 14 days</td>\n",
       "      <td>DURATION</td>\n",
       "      <td>0.7197667</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Fragmin</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.9837</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>5000 units</td>\n",
       "      <td>DOSAGE</td>\n",
       "      <td>0.69355</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>100 mg</td>\n",
       "      <td>STRENGTH</td>\n",
       "      <td>0.7059</td>\n",
       "      <td>62</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>p.o</td>\n",
       "      <td>ROUTE</td>\n",
       "      <td>0.9987</td>\n",
       "      <td>63</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>daily</td>\n",
       "      <td>FREQUENCY</td>\n",
       "      <td>0.9956</td>\n",
       "      <td>64</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Bactrim DS</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.7446</td>\n",
       "      <td>65</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>b.i.d</td>\n",
       "      <td>FREQUENCY</td>\n",
       "      <td>0.9997</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>67 rows × 7 columns</p>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1cdfd71e-c443-4eff-8438-e057176be8aa')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-1cdfd71e-c443-4eff-8438-e057176be8aa button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-1cdfd71e-c443-4eff-8438-e057176be8aa');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-f1e0893b-13f3-4954-8eff-c1304c6c5779\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f1e0893b-13f3-4954-8eff-c1304c6c5779')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-f1e0893b-13f3-4954-8eff-c1304c6c5779 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                             document entities_posology  \\\n",
       "0   The patient is a 30-year-old female with a lon...           insulin   \n",
       "0   The patient is a 30-year-old female with a lon...           Bactrim   \n",
       "0   The patient is a 30-year-old female with a lon...       for 14 days   \n",
       "0   The patient is a 30-year-old female with a lon...           Fragmin   \n",
       "0   The patient is a 30-year-old female with a lon...        5000 units   \n",
       "..                                                ...               ...   \n",
       "0   The patient is a 30-year-old female with a lon...            100 mg   \n",
       "0   The patient is a 30-year-old female with a lon...               p.o   \n",
       "0   The patient is a 30-year-old female with a lon...             daily   \n",
       "0   The patient is a 30-year-old female with a lon...        Bactrim DS   \n",
       "0   The patient is a 30-year-old female with a lon...             b.i.d   \n",
       "\n",
       "   entities_posology_class entities_posology_confidence  \\\n",
       "0                     DRUG                       0.9858   \n",
       "0                     DRUG                       0.9964   \n",
       "0                 DURATION                    0.7197667   \n",
       "0                     DRUG                       0.9837   \n",
       "0                   DOSAGE                      0.69355   \n",
       "..                     ...                          ...   \n",
       "0                 STRENGTH                       0.7059   \n",
       "0                    ROUTE                       0.9987   \n",
       "0                FREQUENCY                       0.9956   \n",
       "0                     DRUG                       0.7446   \n",
       "0                FREQUENCY                       0.9997   \n",
       "\n",
       "   entities_posology_origin_chunk entities_posology_origin_sentence  \\\n",
       "0                               0                                 0   \n",
       "0                               1                                 0   \n",
       "0                               2                                 0   \n",
       "0                               3                                 0   \n",
       "0                               4                                 0   \n",
       "..                            ...                               ...   \n",
       "0                              62                                 0   \n",
       "0                              63                                 0   \n",
       "0                              64                                 0   \n",
       "0                              65                                 0   \n",
       "0                              66                                 0   \n",
       "\n",
       "                                 word_embedding_glove  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "..                                                ...  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "\n",
       "[67 rows x 7 columns]"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.posology').predict(\"The patient is a 30-year-old female with a long history of insulin dependent diabetes, type 2; coronary artery disease; chronic renal insufficiency; peripheral vascular disease, also secondary to diabetes; who was originally admitted to an outside hospital for what appeared to be acute paraplegia, lower extremities. She did receive a course of Bactrim for 14 days for UTI. Evidently, at some point in time, the patient was noted to develop a pressure-type wound on the sole of her left foot and left great toe. She was also noted to have a large sacral wound; this is in a similar location with her previous laminectomy, and this continues to receive daily care. The patient was transferred secondary to inability to participate in full physical and occupational therapy and continue medical management of her diabetes, the sacral decubitus, left foot pressure wound, and associated complications of diabetes. She is given Fragmin 5000 units subcutaneously daily, Xenaderm to wounds topically b.i.d., Lantus 40 units subcutaneously at bedtime, OxyContin 30 mg p.o. q.12 h., folic acid 1 mg daily, levothyroxine 0.1 mg p.o. daily, Prevacid 30 mg daily, Avandia 4 mg daily, Norvasc 10 mg daily, Lexapro 20 mg daily, aspirin 81 mg daily, Senna 2 tablets p.o. q.a.m., Neurontin 400 mg p.o. t.i.d., Percocet 5/325 mg 2 tablets q.4 h. p.r.n., magnesium citrate 1 bottle p.o. p.r.n., sliding scale coverage insulin, Wellbutrin 100 mg p.o. daily, and Bactrim DS b.i.d.\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WBr9PPeSYaos"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.posology.biobert\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_biobert_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 334
    },
    "id": "OoKV6B33Yh6B",
    "outputId": "28824e7b-64fd-4607-f864-c40da57a5323"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_posology_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-11d87ec0-e412-48a8-acce-e6cae7ff6a66\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_posology</th>\n",
       "      <th>entities_posology_class</th>\n",
       "      <th>entities_posology_confidence</th>\n",
       "      <th>entities_posology_origin_chunk</th>\n",
       "      <th>entities_posology_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>insulin</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.9848</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.7802197933197021, -0.9052895307540894, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Bactrim</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.7802197933197021, -0.9052895307540894, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>for 14 days</td>\n",
       "      <td>DURATION</td>\n",
       "      <td>0.999</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.7802197933197021, -0.9052895307540894, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-11d87ec0-e412-48a8-acce-e6cae7ff6a66')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-11d87ec0-e412-48a8-acce-e6cae7ff6a66 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-11d87ec0-e412-48a8-acce-e6cae7ff6a66');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-f255ba7c-2479-4e02-9899-863a6ae6316b\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f255ba7c-2479-4e02-9899-863a6ae6316b')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-f255ba7c-2479-4e02-9899-863a6ae6316b button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_posology  \\\n",
       "0  The patient is a 30-year-old female with a lon...           insulin   \n",
       "0  The patient is a 30-year-old female with a lon...           Bactrim   \n",
       "0  The patient is a 30-year-old female with a lon...       for 14 days   \n",
       "\n",
       "  entities_posology_class entities_posology_confidence  \\\n",
       "0                    DRUG                       0.9848   \n",
       "0                    DRUG                          1.0   \n",
       "0                DURATION                        0.999   \n",
       "\n",
       "  entities_posology_origin_chunk entities_posology_origin_sentence  \\\n",
       "0                              0                                 0   \n",
       "0                              1                                 0   \n",
       "0                              2                                 0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-0.7802197933197021, -0.9052895307540894, 0....  \n",
       "0  [[-0.7802197933197021, -0.9052895307540894, 0....  \n",
       "0  [[-0.7802197933197021, -0.9052895307540894, 0....  "
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.posology.biobert').predict(\"The patient is a 30-year-old female with a long history of insulin dependent diabetes, type 2; coronary artery disease; chronic renal insufficiency; peripheral vascular disease, also secondary to diabetes; who was originally admitted to an outside hospital for what appeared to be acute paraplegia, lower extremities. She did receive a course of Bactrim for 14 days for UTI. Evidently, at some point in time, the patient was noted to develop a pressure-type wound on the sole of her left foot and left great toe. She was also noted to have a large sacral wound; this is in a similar location with her previous laminectomy, and this continues to receive daily care. The patient was transferred secondary to inability to participate in full physical and occupational therapy and continue medical management of her diabetes, the sacral decubitus, left foot pressure wound, and associated complications of diabetes. She is given Fragmin 5000 units subcutaneously daily, Xenaderm to wounds topically b.i.d., Lantus 40 units subcutaneously at bedtime, OxyContin 30 mg p.o. q.12 h., folic acid 1 mg daily, levothyroxine 0.1 mg p.o. daily, Prevacid 30 mg daily, Avandia 4 mg daily, Norvasc 10 mg daily, Lexapro 20 mg daily, aspirin 81 mg daily, Senna 2 tablets p.o. q.a.m., Neurontin 400 mg p.o. t.i.d., Percocet 5/325 mg 2 tablets q.4 h. p.r.n., magnesium citrate 1 bottle p.o. p.r.n., sliding scale coverage insulin, Wellbutrin 100 mg p.o. daily, and Bactrim DS b.i.d.\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8RG_3kWVYx3M"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.posology.healthcare\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_posology_healthcare_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 736
    },
    "id": "6IYXpcFOY7F0",
    "outputId": "daa758c4-7d0a-490f-95c8-5ad8f74fc64a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_posology_healthcare download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_healthcare_100d download started this may take some time.\n",
      "Approximate size to download 475.8 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-940d7610-c456-4e8c-b1fa-9c24973ad915\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_posology</th>\n",
       "      <th>entities_posology_class</th>\n",
       "      <th>entities_posology_confidence</th>\n",
       "      <th>entities_posology_origin_chunk</th>\n",
       "      <th>entities_posology_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>insulin</td>\n",
       "      <td>Drug</td>\n",
       "      <td>0.9799</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Bactrim</td>\n",
       "      <td>Drug</td>\n",
       "      <td>0.9995</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>for 14 days</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.8392666</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Fragmin</td>\n",
       "      <td>Drug</td>\n",
       "      <td>0.9997</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>5000 units</td>\n",
       "      <td>Dosage</td>\n",
       "      <td>0.68655</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>p.o</td>\n",
       "      <td>Route</td>\n",
       "      <td>0.999</td>\n",
       "      <td>63</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>daily</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.679</td>\n",
       "      <td>64</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Bactrim</td>\n",
       "      <td>Drug</td>\n",
       "      <td>0.9997</td>\n",
       "      <td>65</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>DS</td>\n",
       "      <td>Strength</td>\n",
       "      <td>0.7113</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>b.i.d</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.997</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.11429792642593384, 0.18037879467010498, -...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>68 rows × 7 columns</p>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-940d7610-c456-4e8c-b1fa-9c24973ad915')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-940d7610-c456-4e8c-b1fa-9c24973ad915 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-940d7610-c456-4e8c-b1fa-9c24973ad915');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-bf4ab8ea-f50b-4741-8535-a1afbda2525f\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-bf4ab8ea-f50b-4741-8535-a1afbda2525f')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-bf4ab8ea-f50b-4741-8535-a1afbda2525f button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                             document entities_posology  \\\n",
       "0   The patient is a 30-year-old female with a lon...           insulin   \n",
       "0   The patient is a 30-year-old female with a lon...           Bactrim   \n",
       "0   The patient is a 30-year-old female with a lon...       for 14 days   \n",
       "0   The patient is a 30-year-old female with a lon...           Fragmin   \n",
       "0   The patient is a 30-year-old female with a lon...        5000 units   \n",
       "..                                                ...               ...   \n",
       "0   The patient is a 30-year-old female with a lon...               p.o   \n",
       "0   The patient is a 30-year-old female with a lon...             daily   \n",
       "0   The patient is a 30-year-old female with a lon...           Bactrim   \n",
       "0   The patient is a 30-year-old female with a lon...                DS   \n",
       "0   The patient is a 30-year-old female with a lon...             b.i.d   \n",
       "\n",
       "   entities_posology_class entities_posology_confidence  \\\n",
       "0                     Drug                       0.9799   \n",
       "0                     Drug                       0.9995   \n",
       "0                 Duration                    0.8392666   \n",
       "0                     Drug                       0.9997   \n",
       "0                   Dosage                      0.68655   \n",
       "..                     ...                          ...   \n",
       "0                    Route                        0.999   \n",
       "0                Frequency                        0.679   \n",
       "0                     Drug                       0.9997   \n",
       "0                 Strength                       0.7113   \n",
       "0                Frequency                        0.997   \n",
       "\n",
       "   entities_posology_origin_chunk entities_posology_origin_sentence  \\\n",
       "0                               0                                 0   \n",
       "0                               1                                 0   \n",
       "0                               2                                 0   \n",
       "0                               3                                 0   \n",
       "0                               4                                 0   \n",
       "..                            ...                               ...   \n",
       "0                              63                                 0   \n",
       "0                              64                                 0   \n",
       "0                              65                                 0   \n",
       "0                              66                                 0   \n",
       "0                              67                                 0   \n",
       "\n",
       "                                 word_embedding_glove  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "..                                                ...  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "0   [[-0.11429792642593384, 0.18037879467010498, -...  \n",
       "\n",
       "[68 rows x 7 columns]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.posology.healthcare').predict(\"The patient is a 30-year-old female with a long history of insulin dependent diabetes, type 2; coronary artery disease; chronic renal insufficiency; peripheral vascular disease, also secondary to diabetes; who was originally admitted to an outside hospital for what appeared to be acute paraplegia, lower extremities. She did receive a course of Bactrim for 14 days for UTI. Evidently, at some point in time, the patient was noted to develop a pressure-type wound on the sole of her left foot and left great toe. She was also noted to have a large sacral wound; this is in a similar location with her previous laminectomy, and this continues to receive daily care. The patient was transferred secondary to inability to participate in full physical and occupational therapy and continue medical management of her diabetes, the sacral decubitus, left foot pressure wound, and associated complications of diabetes. She is given Fragmin 5000 units subcutaneously daily, Xenaderm to wounds topically b.i.d., Lantus 40 units subcutaneously at bedtime, OxyContin 30 mg p.o. q.12 h., folic acid 1 mg daily, levothyroxine 0.1 mg p.o. daily, Prevacid 30 mg daily, Avandia 4 mg daily, Norvasc 10 mg daily, Lexapro 20 mg daily, aspirin 81 mg daily, Senna 2 tablets p.o. q.a.m., Neurontin 400 mg p.o. t.i.d., Percocet 5/325 mg 2 tablets q.4 h. p.r.n., magnesium citrate 1 bottle p.o. p.r.n., sliding scale coverage insulin, Wellbutrin 100 mg p.o. daily, and Bactrim DS b.i.d.\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hIaKHlncZIqc"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.posology.small\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_posology_small_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 736
    },
    "id": "uAcOj6TSZV38",
    "outputId": "f49dc65e-b6cc-4ef6-b808-845eddbf18c0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_posology_small download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-88c3df61-ef46-4f14-9826-2396f0245f45\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_posology</th>\n",
       "      <th>entities_posology_class</th>\n",
       "      <th>entities_posology_confidence</th>\n",
       "      <th>entities_posology_origin_chunk</th>\n",
       "      <th>entities_posology_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>insulin</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.995</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Bactrim</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.9997</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>for 14 days</td>\n",
       "      <td>DURATION</td>\n",
       "      <td>0.9154334</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Fragmin</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.9993</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>5000 units</td>\n",
       "      <td>DOSAGE</td>\n",
       "      <td>0.8466</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>p.o.</td>\n",
       "      <td>ROUTE</td>\n",
       "      <td>0.9276</td>\n",
       "      <td>63</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>daily</td>\n",
       "      <td>FREQUENCY</td>\n",
       "      <td>0.9832</td>\n",
       "      <td>64</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>Bactrim</td>\n",
       "      <td>DRUG</td>\n",
       "      <td>0.9992</td>\n",
       "      <td>65</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>DS</td>\n",
       "      <td>STRENGTH</td>\n",
       "      <td>0.6437</td>\n",
       "      <td>66</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 30-year-old female with a lon...</td>\n",
       "      <td>b.i.d</td>\n",
       "      <td>FREQUENCY</td>\n",
       "      <td>0.9979</td>\n",
       "      <td>67</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>68 rows × 7 columns</p>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-88c3df61-ef46-4f14-9826-2396f0245f45')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-88c3df61-ef46-4f14-9826-2396f0245f45 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-88c3df61-ef46-4f14-9826-2396f0245f45');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-f889553b-4d04-41be-b5f8-23214c53e1cf\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-f889553b-4d04-41be-b5f8-23214c53e1cf')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-f889553b-4d04-41be-b5f8-23214c53e1cf button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                             document entities_posology  \\\n",
       "0   The patient is a 30-year-old female with a lon...           insulin   \n",
       "0   The patient is a 30-year-old female with a lon...           Bactrim   \n",
       "0   The patient is a 30-year-old female with a lon...       for 14 days   \n",
       "0   The patient is a 30-year-old female with a lon...           Fragmin   \n",
       "0   The patient is a 30-year-old female with a lon...        5000 units   \n",
       "..                                                ...               ...   \n",
       "0   The patient is a 30-year-old female with a lon...              p.o.   \n",
       "0   The patient is a 30-year-old female with a lon...             daily   \n",
       "0   The patient is a 30-year-old female with a lon...           Bactrim   \n",
       "0   The patient is a 30-year-old female with a lon...                DS   \n",
       "0   The patient is a 30-year-old female with a lon...             b.i.d   \n",
       "\n",
       "   entities_posology_class entities_posology_confidence  \\\n",
       "0                     DRUG                        0.995   \n",
       "0                     DRUG                       0.9997   \n",
       "0                 DURATION                    0.9154334   \n",
       "0                     DRUG                       0.9993   \n",
       "0                   DOSAGE                       0.8466   \n",
       "..                     ...                          ...   \n",
       "0                    ROUTE                       0.9276   \n",
       "0                FREQUENCY                       0.9832   \n",
       "0                     DRUG                       0.9992   \n",
       "0                 STRENGTH                       0.6437   \n",
       "0                FREQUENCY                       0.9979   \n",
       "\n",
       "   entities_posology_origin_chunk entities_posology_origin_sentence  \\\n",
       "0                               0                                 0   \n",
       "0                               1                                 0   \n",
       "0                               2                                 0   \n",
       "0                               3                                 0   \n",
       "0                               4                                 0   \n",
       "..                            ...                               ...   \n",
       "0                              63                                 0   \n",
       "0                              64                                 0   \n",
       "0                              65                                 0   \n",
       "0                              66                                 0   \n",
       "0                              67                                 0   \n",
       "\n",
       "                                 word_embedding_glove  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "..                                                ...  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0   [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "\n",
       "[68 rows x 7 columns]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.posology.small').predict(\"The patient is a 30-year-old female with a long history of insulin dependent diabetes, type 2; coronary artery disease; chronic renal insufficiency; peripheral vascular disease, also secondary to diabetes; who was originally admitted to an outside hospital for what appeared to be acute paraplegia, lower extremities. She did receive a course of Bactrim for 14 days for UTI. Evidently, at some point in time, the patient was noted to develop a pressure-type wound on the sole of her left foot and left great toe. She was also noted to have a large sacral wound; this is in a similar location with her previous laminectomy, and this continues to receive daily care. The patient was transferred secondary to inability to participate in full physical and occupational therapy and continue medical management of her diabetes, the sacral decubitus, left foot pressure wound, and associated complications of diabetes. She is given Fragmin 5000 units subcutaneously daily, Xenaderm to wounds topically b.i.d., Lantus 40 units subcutaneously at bedtime, OxyContin 30 mg p.o. q.12 h., folic acid 1 mg daily, levothyroxine 0.1 mg p.o. daily, Prevacid 30 mg daily, Avandia 4 mg daily, Norvasc 10 mg daily, Lexapro 20 mg daily, aspirin 81 mg daily, Senna 2 tablets p.o. q.a.m., Neurontin 400 mg p.o. t.i.d., Percocet 5/325 mg 2 tablets q.4 h. p.r.n., magnesium citrate 1 bottle p.o. p.r.n., sliding scale coverage insulin, Wellbutrin 100 mg p.o. daily, and Bactrim DS b.i.d.\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5vv-xV69Zg2c"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.radiology\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_radiology_en.html)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 981
    },
    "id": "CfDEZjh0ZtBj",
    "outputId": "f69cbac7-bd79-4a2b-8a78-63a8590a869e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_radiology download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-aba93b90-529d-411b-90a3-8ec07e3991a0\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_radiology</th>\n",
       "      <th>entities_radiology_class</th>\n",
       "      <th>entities_radiology_confidence</th>\n",
       "      <th>entities_radiology_origin_chunk</th>\n",
       "      <th>entities_radiology_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>Bilateral breast</td>\n",
       "      <td>BodyPart</td>\n",
       "      <td>0.94865</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>ultrasound</td>\n",
       "      <td>ImagingTest</td>\n",
       "      <td>0.6516</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>ovoid mass</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.6041</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>0.5 x 0.5 x 0.4</td>\n",
       "      <td>Measurements</td>\n",
       "      <td>0.9752</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>cm</td>\n",
       "      <td>Units</td>\n",
       "      <td>0.9894</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>anteromedial aspect of the left shoulder</td>\n",
       "      <td>BodyPart</td>\n",
       "      <td>0.7248333</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>mass</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.942</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>isoechoic echotexture</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.5994</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>muscle</td>\n",
       "      <td>BodyPart</td>\n",
       "      <td>0.7415</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>internal color flow</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.3593</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>benign fibrous tissue</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.38586667</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>lipoma</td>\n",
       "      <td>Disease_Syndrome_Disorder</td>\n",
       "      <td>0.7789</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-aba93b90-529d-411b-90a3-8ec07e3991a0')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-aba93b90-529d-411b-90a3-8ec07e3991a0 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-aba93b90-529d-411b-90a3-8ec07e3991a0');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-84d7736c-eed2-43c4-a92e-eb2f0f83e8f9\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-84d7736c-eed2-43c4-a92e-eb2f0f83e8f9')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-84d7736c-eed2-43c4-a92e-eb2f0f83e8f9 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "0  Bilateral breast ultrasound was subsequently p...   \n",
       "\n",
       "                         entities_radiology   entities_radiology_class  \\\n",
       "0                          Bilateral breast                   BodyPart   \n",
       "0                                ultrasound                ImagingTest   \n",
       "0                                ovoid mass            ImagingFindings   \n",
       "0                           0.5 x 0.5 x 0.4               Measurements   \n",
       "0                                        cm                      Units   \n",
       "0  anteromedial aspect of the left shoulder                   BodyPart   \n",
       "0                                      mass            ImagingFindings   \n",
       "0                     isoechoic echotexture            ImagingFindings   \n",
       "0                                    muscle                   BodyPart   \n",
       "0                       internal color flow            ImagingFindings   \n",
       "0                     benign fibrous tissue            ImagingFindings   \n",
       "0                                    lipoma  Disease_Syndrome_Disorder   \n",
       "\n",
       "  entities_radiology_confidence entities_radiology_origin_chunk  \\\n",
       "0                       0.94865                               0   \n",
       "0                        0.6516                               1   \n",
       "0                        0.6041                               2   \n",
       "0                        0.9752                               3   \n",
       "0                        0.9894                               4   \n",
       "0                     0.7248333                               5   \n",
       "0                         0.942                               6   \n",
       "0                        0.5994                               7   \n",
       "0                        0.7415                               8   \n",
       "0                        0.3593                               9   \n",
       "0                    0.38586667                              10   \n",
       "0                        0.7789                              11   \n",
       "\n",
       "  entities_radiology_origin_sentence  \\\n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "0                                  0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.radiology').predict(\"Bilateral breast ultrasound was subsequently performed, which demonstrated an ovoid mass measuring approximately 0.5 x 0.5 x 0.4 cm in diameter located within the anteromedial aspect of the left shoulder. This mass demonstrates isoechoic echotexture to the adjacent muscle, with no evidence of internal color flow. This may represent benign fibrous tissue or a lipoma.\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "E5Zu6cupZ8Z4"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.radiology.wip_clinical\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_radiology_wip_clinical_en.html\t)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 919
    },
    "id": "iCNw3nzpaKoI",
    "outputId": "0b3a86a6-4842-4068-8f68-4cf599155914"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_radiology_wip_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-047f0eeb-0fa6-40e5-b222-18941fe33d36\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_radiology</th>\n",
       "      <th>entities_radiology_class</th>\n",
       "      <th>entities_radiology_confidence</th>\n",
       "      <th>entities_radiology_origin_chunk</th>\n",
       "      <th>entities_radiology_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>Bilateral</td>\n",
       "      <td>Direction</td>\n",
       "      <td>0.985</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>breast</td>\n",
       "      <td>BodyPart</td>\n",
       "      <td>0.867</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>ultrasound</td>\n",
       "      <td>ImagingTest</td>\n",
       "      <td>0.6285</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>ovoid mass</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.69535</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>0.5 x 0.5 x 0.4</td>\n",
       "      <td>Measurements</td>\n",
       "      <td>0.92204</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>cm</td>\n",
       "      <td>Units</td>\n",
       "      <td>0.9994</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>anteromedial aspect</td>\n",
       "      <td>BodyPart</td>\n",
       "      <td>0.79735005</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>left</td>\n",
       "      <td>Direction</td>\n",
       "      <td>0.4409</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>shoulder</td>\n",
       "      <td>BodyPart</td>\n",
       "      <td>0.6458</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>mass</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.9564</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>isoechoic echotexture</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.54645</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>muscle</td>\n",
       "      <td>BodyPart</td>\n",
       "      <td>0.7968</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>internal color flow</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.47743332</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>benign fibrous tissue</td>\n",
       "      <td>ImagingFindings</td>\n",
       "      <td>0.47533333</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Bilateral breast ultrasound was subsequently p...</td>\n",
       "      <td>lipoma</td>\n",
       "      <td>Disease_Syndrome_Disorder</td>\n",
       "      <td>0.7462</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.5400417447090149, 0.24526117742061615, -0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-047f0eeb-0fa6-40e5-b222-18941fe33d36')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-047f0eeb-0fa6-40e5-b222-18941fe33d36 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-047f0eeb-0fa6-40e5-b222-18941fe33d36');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-d72201fb-9b1a-4b3d-9ffe-f3c108441710\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d72201fb-9b1a-4b3d-9ffe-f3c108441710')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-d72201fb-9b1a-4b3d-9ffe-f3c108441710 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document     entities_radiology  \\\n",
       "0  Bilateral breast ultrasound was subsequently p...              Bilateral   \n",
       "0  Bilateral breast ultrasound was subsequently p...                 breast   \n",
       "0  Bilateral breast ultrasound was subsequently p...             ultrasound   \n",
       "0  Bilateral breast ultrasound was subsequently p...             ovoid mass   \n",
       "0  Bilateral breast ultrasound was subsequently p...        0.5 x 0.5 x 0.4   \n",
       "0  Bilateral breast ultrasound was subsequently p...                     cm   \n",
       "0  Bilateral breast ultrasound was subsequently p...    anteromedial aspect   \n",
       "0  Bilateral breast ultrasound was subsequently p...                   left   \n",
       "0  Bilateral breast ultrasound was subsequently p...               shoulder   \n",
       "0  Bilateral breast ultrasound was subsequently p...                   mass   \n",
       "0  Bilateral breast ultrasound was subsequently p...  isoechoic echotexture   \n",
       "0  Bilateral breast ultrasound was subsequently p...                 muscle   \n",
       "0  Bilateral breast ultrasound was subsequently p...    internal color flow   \n",
       "0  Bilateral breast ultrasound was subsequently p...  benign fibrous tissue   \n",
       "0  Bilateral breast ultrasound was subsequently p...                 lipoma   \n",
       "\n",
       "    entities_radiology_class entities_radiology_confidence  \\\n",
       "0                  Direction                         0.985   \n",
       "0                   BodyPart                         0.867   \n",
       "0                ImagingTest                        0.6285   \n",
       "0            ImagingFindings                       0.69535   \n",
       "0               Measurements                       0.92204   \n",
       "0                      Units                        0.9994   \n",
       "0                   BodyPart                    0.79735005   \n",
       "0                  Direction                        0.4409   \n",
       "0                   BodyPart                        0.6458   \n",
       "0            ImagingFindings                        0.9564   \n",
       "0            ImagingFindings                       0.54645   \n",
       "0                   BodyPart                        0.7968   \n",
       "0            ImagingFindings                    0.47743332   \n",
       "0            ImagingFindings                    0.47533333   \n",
       "0  Disease_Syndrome_Disorder                        0.7462   \n",
       "\n",
       "  entities_radiology_origin_chunk entities_radiology_origin_sentence  \\\n",
       "0                               0                                  0   \n",
       "0                               1                                  0   \n",
       "0                               2                                  0   \n",
       "0                               3                                  0   \n",
       "0                               4                                  0   \n",
       "0                               5                                  0   \n",
       "0                               6                                  0   \n",
       "0                               7                                  0   \n",
       "0                               8                                  0   \n",
       "0                               9                                  0   \n",
       "0                              10                                  0   \n",
       "0                              11                                  0   \n",
       "0                              12                                  0   \n",
       "0                              13                                  0   \n",
       "0                              14                                  0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  \n",
       "0  [[-0.5400417447090149, 0.24526117742061615, -0...  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.radiology.wip_clinical').predict(\"Bilateral breast ultrasound was subsequently performed, which demonstrated an ovoid mass measuring approximately 0.5 x 0.5 x 0.4 cm in diameter located within the anteromedial aspect of the left shoulder. This mass demonstrates isoechoic echotexture to the adjacent muscle, with no evidence of internal color flow. This may represent benign fibrous tissue or a lipoma.\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "B512tqR0atP-"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.risk_factors\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_risk_factors_en.html\t)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 522
    },
    "id": "NW6wd6_XbLxR",
    "outputId": "6c6ab178-7f5a-43ba-ceb5-105ebbe786e5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_risk_factors download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-a27656d7-54ae-4370-9944-658ff9055011\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_risk_factors</th>\n",
       "      <th>entities_risk_factors_class</th>\n",
       "      <th>entities_risk_factors_confidence</th>\n",
       "      <th>entities_risk_factors_origin_chunk</th>\n",
       "      <th>entities_risk_factors_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>chest pain</td>\n",
       "      <td>CAD</td>\n",
       "      <td>0.7395</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>diabetic</td>\n",
       "      <td>DIABETES</td>\n",
       "      <td>0.9999</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>coronary artery disease</td>\n",
       "      <td>CAD</td>\n",
       "      <td>0.7496333</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>1995</td>\n",
       "      <td>PHI</td>\n",
       "      <td>0.984</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a27656d7-54ae-4370-9944-658ff9055011')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-a27656d7-54ae-4370-9944-658ff9055011 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-a27656d7-54ae-4370-9944-658ff9055011');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-4fef9b86-857c-4c3b-9efe-baad68c3b87a\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4fef9b86-857c-4c3b-9efe-baad68c3b87a')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-4fef9b86-857c-4c3b-9efe-baad68c3b87a button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document    entities_risk_factors  \\\n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...               chest pain   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...                 diabetic   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...  coronary artery disease   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...                     1995   \n",
       "\n",
       "  entities_risk_factors_class entities_risk_factors_confidence  \\\n",
       "0                         CAD                           0.7395   \n",
       "0                    DIABETES                           0.9999   \n",
       "0                         CAD                        0.7496333   \n",
       "0                         PHI                            0.984   \n",
       "\n",
       "  entities_risk_factors_origin_chunk entities_risk_factors_origin_sentence  \\\n",
       "0                                  0                                     0   \n",
       "0                                  1                                     0   \n",
       "0                                  2                                     0   \n",
       "0                                  3                                     0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.risk_factors').predict('HISTORY OF PRESENT ILLNESS: The patient is a 40-year-old white male who presents with a chief complaint of \"chest pain\". The patient is diabetic and has a prior history of coronary artery disease. The patient presents today stating that his chest pain started yesterday evening and has been somewhat intermittent. The severity of the pain has progressively increased. He describes the pain as a sharp and heavy pain which radiates to his neck & left arm. He ranks the pain a 7 on a scale of 1-10. He admits some shortness of breath & diaphoresis. He states that he has had nausea & 3 episodes of vomiting tonight. He denies any fever or chills. He admits prior episodes of similar pain prior to his PTCA in 1995. He states the pain is somewhat worse with walking and seems to be relieved with rest. There is no change in pain with positioning. He states that he took 3 nitroglycerin tablets sublingually over the past 1 hour, which he states has partially relieved his pain. The patient ranks his present pain a 4 on a scale of 1-10. The most recent episode of pain has lasted one-hour. The patient denies any history of recent surgery, head trauma, recent stroke, abnormal bleeding such as blood in urine or stool or nosebleed.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "szi-leEbbwJ5"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.risk_factors.biobert\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_risk_factors_biobert_en.html\t)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 390
    },
    "id": "ffqCbuu5b7k8",
    "outputId": "17cb5fe7-1dd8-4f04-e3f1-12c007d350db"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_risk_factors_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-1917c148-e467-48a4-bc4e-964d01f9d457\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_risk_factors</th>\n",
       "      <th>entities_risk_factors_class</th>\n",
       "      <th>entities_risk_factors_confidence</th>\n",
       "      <th>entities_risk_factors_origin_chunk</th>\n",
       "      <th>entities_risk_factors_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>diabetic</td>\n",
       "      <td>DIABETES</td>\n",
       "      <td>0.9845</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.205116868019104, 0.35607460141181946, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>prior history of coronary artery disease</td>\n",
       "      <td>CAD</td>\n",
       "      <td>0.44113335</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.205116868019104, 0.35607460141181946, -0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1917c148-e467-48a4-bc4e-964d01f9d457')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-1917c148-e467-48a4-bc4e-964d01f9d457 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-1917c148-e467-48a4-bc4e-964d01f9d457');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-68512e48-87fb-42c3-b4eb-dea8cdce2727\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-68512e48-87fb-42c3-b4eb-dea8cdce2727')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-68512e48-87fb-42c3-b4eb-dea8cdce2727 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "\n",
       "                      entities_risk_factors entities_risk_factors_class  \\\n",
       "0                                  diabetic                    DIABETES   \n",
       "0  prior history of coronary artery disease                         CAD   \n",
       "\n",
       "  entities_risk_factors_confidence entities_risk_factors_origin_chunk  \\\n",
       "0                           0.9845                                  0   \n",
       "0                       0.44113335                                  1   \n",
       "\n",
       "  entities_risk_factors_origin_sentence  \\\n",
       "0                                     0   \n",
       "0                                     0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-1.205116868019104, 0.35607460141181946, -0....  \n",
       "0  [[-1.205116868019104, 0.35607460141181946, -0....  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.risk_factors.biobert').predict('HISTORY OF PRESENT ILLNESS: The patient is a 40-year-old white male who presents with a chief complaint of \"chest pain\". The patient is diabetic and has a prior history of coronary artery disease. The patient presents today stating that his chest pain started yesterday evening and has been somewhat intermittent. The severity of the pain has progressively increased. He describes the pain as a sharp and heavy pain which radiates to his neck & left arm. He ranks the pain a 7 on a scale of 1-10. He admits some shortness of breath & diaphoresis. He states that he has had nausea & 3 episodes of vomiting tonight. He denies any fever or chills. He admits prior episodes of similar pain prior to his PTCA in 1995. He states the pain is somewhat worse with walking and seems to be relieved with rest. There is no change in pain with positioning. He states that he took 3 nitroglycerin tablets sublingually over the past 1 hour, which he states has partially relieved his pain. The patient ranks his present pain a 4 on a scale of 1-10. The most recent episode of pain has lasted one-hour. The patient denies any history of recent surgery, head trauma, recent stroke, abnormal bleeding such as blood in urine or stool or nosebleed.', output_level='chunk')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1W1f8NF9ciaA"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.tumour\t\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/nerdl_tumour_demo_en.html\t)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 220
    },
    "id": "TQZvJQgAcz7_",
    "outputId": "3e42a647-1aeb-468c-9869-0c04f9aa6708"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "nerdl_tumour_demo download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-a4be04e2-262b-4e28-a3e8-8a4e8c32c1c4\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_tumour</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a4be04e2-262b-4e28-a3e8-8a4e8c32c1c4')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-a4be04e2-262b-4e28-a3e8-8a4e8c32c1c4 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-a4be04e2-262b-4e28-a3e8-8a4e8c32c1c4');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_tumour  \\\n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...             NaN   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.tumour').predict('HISTORY OF PRESENT ILLNESS: The patient is a 40-year-old white male who presents with a chief complaint of \"chest pain\". The patient is diabetic and has a prior history of coronary artery disease. The patient presents today stating that his chest pain started yesterday evening and has been somewhat intermittent. The severity of the pain has progressively increased. He describes the pain as a sharp and heavy pain which radiates to his neck & left arm. He ranks the pain a 7 on a scale of 1-10. He admits some shortness of breath & diaphoresis. He states that he has had nausea & 3 episodes of vomiting tonight. He denies any fever or chills. He admits prior episodes of similar pain prior to his PTCA in 1995. He states the pain is somewhat worse with walking and seems to be relieved with rest. There is no change in pain with positioning. He states that he took 3 nitroglycerin tablets sublingually over the past 1 hour, which he states has partially relieved his pain. The patient ranks his present pain a 4 on a scale of 1-10. The most recent episode of pain has lasted one-hour. The patient denies any history of recent surgery, head trauma, recent stroke, abnormal bleeding such as blood in urine or stool or nosebleed.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Q1u9b84pdPs5"
   },
   "source": [
    "# [Medical Named Entity Recognizer:  ner.jsl.wip.clinical.greedy\t\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/jsl_ner_wip_clinical_en.html\t)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "Bs_cikX5ddpZ",
    "outputId": "7ba8adff-3707-4e9d-de7a-4e3a644264ca"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "jsl_ner_wip_greedy_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-c99f97a4-4166-45fc-acfd-c7346d23e221\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_jsl</th>\n",
       "      <th>entities_jsl_class</th>\n",
       "      <th>entities_jsl_confidence</th>\n",
       "      <th>entities_jsl_origin_chunk</th>\n",
       "      <th>entities_jsl_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>21-day-old</td>\n",
       "      <td>Age</td>\n",
       "      <td>0.9822</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>Race_Ethnicity</td>\n",
       "      <td>0.9999</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>male</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9923</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>for 2 days</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.69556665</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8771</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.8069</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>suctioning yellow discharge</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.27296665</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>nares</td>\n",
       "      <td>External_body_part_or_region</td>\n",
       "      <td>0.6004</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>she</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9964</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mild problems with his breathing while feeding</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.44225714</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>perioral cyanosis</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.34745</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>retractions</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9465</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>One day ago</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.3499</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.7424</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tactile temperature</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.16194999</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Tylenol</td>\n",
       "      <td>Drug</td>\n",
       "      <td>0.9877</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>decreased p.o. intake</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.343875</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9542</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>20 minutes</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.1682</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>q.2h.</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.662</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>to 5 to 10 minutes</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.12748</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.989</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>respiratory congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.4769</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.993</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tired</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8756</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>fussy</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8323</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>over the past 2 days</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.32277998</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>albuterol</td>\n",
       "      <td>Drug</td>\n",
       "      <td>0.9381</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>ER</td>\n",
       "      <td>Clinical_Dept</td>\n",
       "      <td>0.9887</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9715</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>urine output has also decreased</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.21041998</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9884</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>per 24 hours</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.5335</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9947</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>per 24 hours</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.4779</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.739</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>diarrhea</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9762</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9847</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>bowel movements are yellow colored</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.37742</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c99f97a4-4166-45fc-acfd-c7346d23e221')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-c99f97a4-4166-45fc-acfd-c7346d23e221 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-c99f97a4-4166-45fc-acfd-c7346d23e221');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-239f3aa0-f36d-4ee9-949e-f2c3fc746951\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-239f3aa0-f36d-4ee9-949e-f2c3fc746951')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-239f3aa0-f36d-4ee9-949e-f2c3fc746951 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "\n",
       "                                     entities_jsl  \\\n",
       "0                                      21-day-old   \n",
       "0                                       Caucasian   \n",
       "0                                            male   \n",
       "0                                      for 2 days   \n",
       "0                                      congestion   \n",
       "0                                             mom   \n",
       "0                     suctioning yellow discharge   \n",
       "0                                           nares   \n",
       "0                                             she   \n",
       "0  mild problems with his breathing while feeding   \n",
       "0                               perioral cyanosis   \n",
       "0                                     retractions   \n",
       "0                                     One day ago   \n",
       "0                                             mom   \n",
       "0                             tactile temperature   \n",
       "0                                         Tylenol   \n",
       "0                           decreased p.o. intake   \n",
       "0                                             His   \n",
       "0                                      20 minutes   \n",
       "0                                           q.2h.   \n",
       "0                              to 5 to 10 minutes   \n",
       "0                                             his   \n",
       "0                          respiratory congestion   \n",
       "0                                              He   \n",
       "0                                           tired   \n",
       "0                                           fussy   \n",
       "0                            over the past 2 days   \n",
       "0                                       albuterol   \n",
       "0                                              ER   \n",
       "0                                             His   \n",
       "0                 urine output has also decreased   \n",
       "0                                              he   \n",
       "0                                    per 24 hours   \n",
       "0                                              he   \n",
       "0                                    per 24 hours   \n",
       "0                                             Mom   \n",
       "0                                        diarrhea   \n",
       "0                                             His   \n",
       "0              bowel movements are yellow colored   \n",
       "\n",
       "             entities_jsl_class entities_jsl_confidence  \\\n",
       "0                           Age                  0.9822   \n",
       "0                Race_Ethnicity                  0.9999   \n",
       "0                        Gender                  0.9923   \n",
       "0                      Duration              0.69556665   \n",
       "0                       Symptom                  0.8771   \n",
       "0                        Gender                  0.8069   \n",
       "0                       Symptom              0.27296665   \n",
       "0  External_body_part_or_region                  0.6004   \n",
       "0                        Gender                  0.9964   \n",
       "0                       Symptom              0.44225714   \n",
       "0                       Symptom                 0.34745   \n",
       "0                       Symptom                  0.9465   \n",
       "0                  RelativeDate                  0.3499   \n",
       "0                        Gender                  0.7424   \n",
       "0                       Symptom              0.16194999   \n",
       "0                          Drug                  0.9877   \n",
       "0                       Symptom                0.343875   \n",
       "0                        Gender                  0.9542   \n",
       "0                      Duration                  0.1682   \n",
       "0                     Frequency                   0.662   \n",
       "0                      Duration                 0.12748   \n",
       "0                        Gender                   0.989   \n",
       "0                       Symptom                  0.4769   \n",
       "0                        Gender                   0.993   \n",
       "0                       Symptom                  0.8756   \n",
       "0                       Symptom                  0.8323   \n",
       "0                  RelativeDate              0.32277998   \n",
       "0                          Drug                  0.9381   \n",
       "0                 Clinical_Dept                  0.9887   \n",
       "0                        Gender                  0.9715   \n",
       "0                       Symptom              0.21041998   \n",
       "0                        Gender                  0.9884   \n",
       "0                     Frequency                  0.5335   \n",
       "0                        Gender                  0.9947   \n",
       "0                     Frequency                  0.4779   \n",
       "0                        Gender                   0.739   \n",
       "0                       Symptom                  0.9762   \n",
       "0                        Gender                  0.9847   \n",
       "0                       Symptom                 0.37742   \n",
       "\n",
       "  entities_jsl_origin_chunk entities_jsl_origin_sentence  \\\n",
       "0                         0                            0   \n",
       "0                         1                            0   \n",
       "0                         2                            0   \n",
       "0                         3                            0   \n",
       "0                         4                            0   \n",
       "0                         5                            0   \n",
       "0                         6                            0   \n",
       "0                         7                            0   \n",
       "0                         8                            0   \n",
       "0                         9                            0   \n",
       "0                        10                            0   \n",
       "0                        11                            0   \n",
       "0                        12                            0   \n",
       "0                        13                            0   \n",
       "0                        14                            0   \n",
       "0                        15                            0   \n",
       "0                        16                            0   \n",
       "0                        17                            0   \n",
       "0                        18                            0   \n",
       "0                        19                            0   \n",
       "0                        20                            0   \n",
       "0                        21                            0   \n",
       "0                        22                            0   \n",
       "0                        23                            0   \n",
       "0                        24                            0   \n",
       "0                        25                            0   \n",
       "0                        26                            0   \n",
       "0                        27                            0   \n",
       "0                        28                            0   \n",
       "0                        29                            0   \n",
       "0                        30                            0   \n",
       "0                        31                            0   \n",
       "0                        32                            0   \n",
       "0                        33                            0   \n",
       "0                        34                            0   \n",
       "0                        35                            0   \n",
       "0                        36                            0   \n",
       "0                        37                            0   \n",
       "0                        38                            0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.jsl.wip.clinical.greedy').predict(\"The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature.\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XgAPfIWEd3AK"
   },
   "source": [
    "# [Medical Named Entity Recognizer: ner.jsl.wip.clinical.modifier\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/jsl_ner_wip_modifier_clinical_en.html\t)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "wik-5ZjCeAM5",
    "outputId": "db50a7da-e174-4844-8e05-84981feac057"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "jsl_ner_wip_modifier_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-b39ee8a3-60e5-41ee-8413-f4b3c8325710\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_jsl</th>\n",
       "      <th>entities_jsl_class</th>\n",
       "      <th>entities_jsl_confidence</th>\n",
       "      <th>entities_jsl_origin_chunk</th>\n",
       "      <th>entities_jsl_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>21-day-old</td>\n",
       "      <td>Age</td>\n",
       "      <td>0.9822</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Caucasian</td>\n",
       "      <td>Race_Ethnicity</td>\n",
       "      <td>0.9999</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>male</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9923</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>for 2 days</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.69556665</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8771</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.8069</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>suctioning yellow discharge</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.27296665</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>nares</td>\n",
       "      <td>External_body_part_or_region</td>\n",
       "      <td>0.6004</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>she</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9964</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mild problems with his breathing while feeding</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.44225714</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>perioral cyanosis</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.34745</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>retractions</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9465</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>One day ago</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.3499</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.7424</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tactile temperature</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.16194999</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Tylenol</td>\n",
       "      <td>Drug</td>\n",
       "      <td>0.9877</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>decreased p.o. intake</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.343875</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9542</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>20 minutes</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.1682</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>q.2h.</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.662</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>to 5 to 10 minutes</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.12748</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.989</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>respiratory congestion</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.4769</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.993</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>tired</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8756</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>fussy</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8323</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>over the past 2 days</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.32277998</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>albuterol</td>\n",
       "      <td>Drug</td>\n",
       "      <td>0.9381</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>ER</td>\n",
       "      <td>Clinical_Dept</td>\n",
       "      <td>0.9887</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9715</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>urine output has also decreased</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.21041998</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9884</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>per 24 hours</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.5335</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9947</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>per 24 hours</td>\n",
       "      <td>Frequency</td>\n",
       "      <td>0.4779</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>Mom</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.739</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>diarrhea</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9762</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>His</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9847</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The patient is a 21-day-old Caucasian male her...</td>\n",
       "      <td>bowel movements are yellow colored</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.37742</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.0823686420917511, -0.3178570866584778, 0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b39ee8a3-60e5-41ee-8413-f4b3c8325710')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-b39ee8a3-60e5-41ee-8413-f4b3c8325710 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-b39ee8a3-60e5-41ee-8413-f4b3c8325710');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-4fe5d64c-33d0-479a-a6a3-3c048e8962d4\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4fe5d64c-33d0-479a-a6a3-3c048e8962d4')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-4fe5d64c-33d0-479a-a6a3-3c048e8962d4 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "0  The patient is a 21-day-old Caucasian male her...   \n",
       "\n",
       "                                     entities_jsl  \\\n",
       "0                                      21-day-old   \n",
       "0                                       Caucasian   \n",
       "0                                            male   \n",
       "0                                      for 2 days   \n",
       "0                                      congestion   \n",
       "0                                             mom   \n",
       "0                     suctioning yellow discharge   \n",
       "0                                           nares   \n",
       "0                                             she   \n",
       "0  mild problems with his breathing while feeding   \n",
       "0                               perioral cyanosis   \n",
       "0                                     retractions   \n",
       "0                                     One day ago   \n",
       "0                                             mom   \n",
       "0                             tactile temperature   \n",
       "0                                         Tylenol   \n",
       "0                           decreased p.o. intake   \n",
       "0                                             His   \n",
       "0                                      20 minutes   \n",
       "0                                           q.2h.   \n",
       "0                              to 5 to 10 minutes   \n",
       "0                                             his   \n",
       "0                          respiratory congestion   \n",
       "0                                              He   \n",
       "0                                           tired   \n",
       "0                                           fussy   \n",
       "0                            over the past 2 days   \n",
       "0                                       albuterol   \n",
       "0                                              ER   \n",
       "0                                             His   \n",
       "0                 urine output has also decreased   \n",
       "0                                              he   \n",
       "0                                    per 24 hours   \n",
       "0                                              he   \n",
       "0                                    per 24 hours   \n",
       "0                                             Mom   \n",
       "0                                        diarrhea   \n",
       "0                                             His   \n",
       "0              bowel movements are yellow colored   \n",
       "\n",
       "             entities_jsl_class entities_jsl_confidence  \\\n",
       "0                           Age                  0.9822   \n",
       "0                Race_Ethnicity                  0.9999   \n",
       "0                        Gender                  0.9923   \n",
       "0                      Duration              0.69556665   \n",
       "0                       Symptom                  0.8771   \n",
       "0                        Gender                  0.8069   \n",
       "0                       Symptom              0.27296665   \n",
       "0  External_body_part_or_region                  0.6004   \n",
       "0                        Gender                  0.9964   \n",
       "0                       Symptom              0.44225714   \n",
       "0                       Symptom                 0.34745   \n",
       "0                       Symptom                  0.9465   \n",
       "0                  RelativeDate                  0.3499   \n",
       "0                        Gender                  0.7424   \n",
       "0                       Symptom              0.16194999   \n",
       "0                          Drug                  0.9877   \n",
       "0                       Symptom                0.343875   \n",
       "0                        Gender                  0.9542   \n",
       "0                      Duration                  0.1682   \n",
       "0                     Frequency                   0.662   \n",
       "0                      Duration                 0.12748   \n",
       "0                        Gender                   0.989   \n",
       "0                       Symptom                  0.4769   \n",
       "0                        Gender                   0.993   \n",
       "0                       Symptom                  0.8756   \n",
       "0                       Symptom                  0.8323   \n",
       "0                  RelativeDate              0.32277998   \n",
       "0                          Drug                  0.9381   \n",
       "0                 Clinical_Dept                  0.9887   \n",
       "0                        Gender                  0.9715   \n",
       "0                       Symptom              0.21041998   \n",
       "0                        Gender                  0.9884   \n",
       "0                     Frequency                  0.5335   \n",
       "0                        Gender                  0.9947   \n",
       "0                     Frequency                  0.4779   \n",
       "0                        Gender                   0.739   \n",
       "0                       Symptom                  0.9762   \n",
       "0                        Gender                  0.9847   \n",
       "0                       Symptom                 0.37742   \n",
       "\n",
       "  entities_jsl_origin_chunk entities_jsl_origin_sentence  \\\n",
       "0                         0                            0   \n",
       "0                         1                            0   \n",
       "0                         2                            0   \n",
       "0                         3                            0   \n",
       "0                         4                            0   \n",
       "0                         5                            0   \n",
       "0                         6                            0   \n",
       "0                         7                            0   \n",
       "0                         8                            0   \n",
       "0                         9                            0   \n",
       "0                        10                            0   \n",
       "0                        11                            0   \n",
       "0                        12                            0   \n",
       "0                        13                            0   \n",
       "0                        14                            0   \n",
       "0                        15                            0   \n",
       "0                        16                            0   \n",
       "0                        17                            0   \n",
       "0                        18                            0   \n",
       "0                        19                            0   \n",
       "0                        20                            0   \n",
       "0                        21                            0   \n",
       "0                        22                            0   \n",
       "0                        23                            0   \n",
       "0                        24                            0   \n",
       "0                        25                            0   \n",
       "0                        26                            0   \n",
       "0                        27                            0   \n",
       "0                        28                            0   \n",
       "0                        29                            0   \n",
       "0                        30                            0   \n",
       "0                        31                            0   \n",
       "0                        32                            0   \n",
       "0                        33                            0   \n",
       "0                        34                            0   \n",
       "0                        35                            0   \n",
       "0                        36                            0   \n",
       "0                        37                            0   \n",
       "0                        38                            0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  \n",
       "0  [[-0.0823686420917511, -0.3178570866584778, 0....  "
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.jsl.wip.clinical.modifier').predict(\"The patient is a 21-day-old Caucasian male here for 2 days of congestion - mom has been suctioning yellow discharge from the patient's nares, plus she has noticed some mild problems with his breathing while feeding (but negative for any perioral cyanosis or retractions). One day ago, mom also noticed a tactile temperature and gave the patient Tylenol. Baby also has had some decreased p.o. intake. His normal breast-feeding is down from 20 minutes q.2h. to 5 to 10 minutes secondary to his respiratory congestion. He sleeps well, but has been more tired and has been fussy over the past 2 days. The parents noticed no improvement with albuterol treatments given in the ER. His urine output has also decreased; normally he has 8 to 10 wet and 5 dirty diapers per 24 hours, now he has down to 4 wet diapers per 24 hours. Mom denies any diarrhea. His bowel movements are yellow colored and soft in nature.\", output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FnJif7kmefdR"
   },
   "source": [
    "# [Medical Named Entity Recognizer: ner.jsl.wip.clinical.rd\t\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/jsl_rd_ner_wip_greedy_clinical_en.html\t)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "ihpSBqn3es3_",
    "outputId": "ad2dbe78-e0bc-4809-c994-d253f1101545"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "jsl_rd_ner_wip_greedy_clinical download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-d9fe9fe1-edce-43c9-9622-dd799e2a3258\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_jsl</th>\n",
       "      <th>entities_jsl_class</th>\n",
       "      <th>entities_jsl_confidence</th>\n",
       "      <th>entities_jsl_origin_chunk</th>\n",
       "      <th>entities_jsl_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>HISTORY OF PRESENT ILLNESS:</td>\n",
       "      <td>Section_Header</td>\n",
       "      <td>0.48834</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>40-year-old</td>\n",
       "      <td>Age</td>\n",
       "      <td>0.9979</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>white</td>\n",
       "      <td>Race_Ethnicity</td>\n",
       "      <td>0.9975</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>male</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9989</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>chest pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.55114996</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>diabetic</td>\n",
       "      <td>Diabetes</td>\n",
       "      <td>0.988</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>coronary artery disease</td>\n",
       "      <td>Heart_Disease</td>\n",
       "      <td>0.58363336</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>today</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.9999</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9963</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>chest pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.4116</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>yesterday</td>\n",
       "      <td>RelativeDate</td>\n",
       "      <td>0.9998</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>evening</td>\n",
       "      <td>RelativeTime</td>\n",
       "      <td>0.9204</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>intermittent</td>\n",
       "      <td>Modifier</td>\n",
       "      <td>0.8442</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9452</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>progressively increased</td>\n",
       "      <td>Modifier</td>\n",
       "      <td>0.72595</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9981</td>\n",
       "      <td>15</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9571</td>\n",
       "      <td>16</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>sharp and heavy pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.47162503</td>\n",
       "      <td>17</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9621</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>neck</td>\n",
       "      <td>BodyPart</td>\n",
       "      <td>0.7772</td>\n",
       "      <td>19</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>left</td>\n",
       "      <td>Direction</td>\n",
       "      <td>0.7792</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>arm</td>\n",
       "      <td>BodyPart</td>\n",
       "      <td>0.4877</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9972</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9398</td>\n",
       "      <td>23</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9988</td>\n",
       "      <td>24</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>shortness of breath</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.5703</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>diaphoresis</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9862</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9993</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9985</td>\n",
       "      <td>28</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>nausea</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9814</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>vomiting</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9592</td>\n",
       "      <td>30</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9971</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>fever</td>\n",
       "      <td>VS_Finding</td>\n",
       "      <td>0.9873</td>\n",
       "      <td>32</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>chills</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.997</td>\n",
       "      <td>33</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9989</td>\n",
       "      <td>34</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>similar pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.626</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9942</td>\n",
       "      <td>36</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>PTCA</td>\n",
       "      <td>Procedure</td>\n",
       "      <td>0.803</td>\n",
       "      <td>37</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>1995</td>\n",
       "      <td>Date</td>\n",
       "      <td>0.98</td>\n",
       "      <td>38</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9988</td>\n",
       "      <td>39</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.958</td>\n",
       "      <td>40</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>change in pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.47293332</td>\n",
       "      <td>41</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>He</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9912</td>\n",
       "      <td>42</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9595</td>\n",
       "      <td>43</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>3 nitroglycerin tablets sublingually</td>\n",
       "      <td>Drug</td>\n",
       "      <td>0.646775</td>\n",
       "      <td>44</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>over the past 1 hour</td>\n",
       "      <td>Duration</td>\n",
       "      <td>0.27346</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>he</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9952</td>\n",
       "      <td>46</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9942</td>\n",
       "      <td>47</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8749</td>\n",
       "      <td>48</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>his</td>\n",
       "      <td>Gender</td>\n",
       "      <td>0.9948</td>\n",
       "      <td>49</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8583</td>\n",
       "      <td>50</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>pain</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.9765</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>head trauma</td>\n",
       "      <td>Injury_or_Poisoning</td>\n",
       "      <td>0.8581</td>\n",
       "      <td>52</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>recent stroke</td>\n",
       "      <td>Cerebrovascular_Disease</td>\n",
       "      <td>0.85845</td>\n",
       "      <td>53</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>abnormal bleeding</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.88325</td>\n",
       "      <td>54</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>blood in urine or stool</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.51497996</td>\n",
       "      <td>55</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: The patient is a 4...</td>\n",
       "      <td>nosebleed</td>\n",
       "      <td>Symptom</td>\n",
       "      <td>0.8984</td>\n",
       "      <td>56</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d9fe9fe1-edce-43c9-9622-dd799e2a3258')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-d9fe9fe1-edce-43c9-9622-dd799e2a3258 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-d9fe9fe1-edce-43c9-9622-dd799e2a3258');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-2468659a-2429-443d-82fc-a0c9e035c1b1\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-2468659a-2429-443d-82fc-a0c9e035c1b1')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-2468659a-2429-443d-82fc-a0c9e035c1b1 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "0  HISTORY OF PRESENT ILLNESS: The patient is a 4...   \n",
       "\n",
       "                           entities_jsl       entities_jsl_class  \\\n",
       "0           HISTORY OF PRESENT ILLNESS:           Section_Header   \n",
       "0                           40-year-old                      Age   \n",
       "0                                 white           Race_Ethnicity   \n",
       "0                                  male                   Gender   \n",
       "0                            chest pain                  Symptom   \n",
       "0                              diabetic                 Diabetes   \n",
       "0               coronary artery disease            Heart_Disease   \n",
       "0                                 today             RelativeDate   \n",
       "0                                   his                   Gender   \n",
       "0                            chest pain                  Symptom   \n",
       "0                             yesterday             RelativeDate   \n",
       "0                               evening             RelativeTime   \n",
       "0                          intermittent                 Modifier   \n",
       "0                                  pain                  Symptom   \n",
       "0               progressively increased                 Modifier   \n",
       "0                                    He                   Gender   \n",
       "0                                  pain                  Symptom   \n",
       "0                  sharp and heavy pain                  Symptom   \n",
       "0                                   his                   Gender   \n",
       "0                                  neck                 BodyPart   \n",
       "0                                  left                Direction   \n",
       "0                                   arm                 BodyPart   \n",
       "0                                    He                   Gender   \n",
       "0                                  pain                  Symptom   \n",
       "0                                    He                   Gender   \n",
       "0                   shortness of breath                  Symptom   \n",
       "0                           diaphoresis                  Symptom   \n",
       "0                                    He                   Gender   \n",
       "0                                    he                   Gender   \n",
       "0                                nausea                  Symptom   \n",
       "0                              vomiting                  Symptom   \n",
       "0                                    He                   Gender   \n",
       "0                                 fever               VS_Finding   \n",
       "0                                chills                  Symptom   \n",
       "0                                    He                   Gender   \n",
       "0                          similar pain                  Symptom   \n",
       "0                                   his                   Gender   \n",
       "0                                  PTCA                Procedure   \n",
       "0                                  1995                     Date   \n",
       "0                                    He                   Gender   \n",
       "0                                  pain                  Symptom   \n",
       "0                        change in pain                  Symptom   \n",
       "0                                    He                   Gender   \n",
       "0                                    he                   Gender   \n",
       "0  3 nitroglycerin tablets sublingually                     Drug   \n",
       "0                  over the past 1 hour                 Duration   \n",
       "0                                    he                   Gender   \n",
       "0                                   his                   Gender   \n",
       "0                                  pain                  Symptom   \n",
       "0                                   his                   Gender   \n",
       "0                                  pain                  Symptom   \n",
       "0                                  pain                  Symptom   \n",
       "0                           head trauma      Injury_or_Poisoning   \n",
       "0                         recent stroke  Cerebrovascular_Disease   \n",
       "0                     abnormal bleeding                  Symptom   \n",
       "0               blood in urine or stool                  Symptom   \n",
       "0                             nosebleed                  Symptom   \n",
       "\n",
       "  entities_jsl_confidence entities_jsl_origin_chunk  \\\n",
       "0                 0.48834                         0   \n",
       "0                  0.9979                         1   \n",
       "0                  0.9975                         2   \n",
       "0                  0.9989                         3   \n",
       "0              0.55114996                         4   \n",
       "0                   0.988                         5   \n",
       "0              0.58363336                         6   \n",
       "0                  0.9999                         7   \n",
       "0                  0.9963                         8   \n",
       "0                  0.4116                         9   \n",
       "0                  0.9998                        10   \n",
       "0                  0.9204                        11   \n",
       "0                  0.8442                        12   \n",
       "0                  0.9452                        13   \n",
       "0                 0.72595                        14   \n",
       "0                  0.9981                        15   \n",
       "0                  0.9571                        16   \n",
       "0              0.47162503                        17   \n",
       "0                  0.9621                        18   \n",
       "0                  0.7772                        19   \n",
       "0                  0.7792                        20   \n",
       "0                  0.4877                        21   \n",
       "0                  0.9972                        22   \n",
       "0                  0.9398                        23   \n",
       "0                  0.9988                        24   \n",
       "0                  0.5703                        25   \n",
       "0                  0.9862                        26   \n",
       "0                  0.9993                        27   \n",
       "0                  0.9985                        28   \n",
       "0                  0.9814                        29   \n",
       "0                  0.9592                        30   \n",
       "0                  0.9971                        31   \n",
       "0                  0.9873                        32   \n",
       "0                   0.997                        33   \n",
       "0                  0.9989                        34   \n",
       "0                   0.626                        35   \n",
       "0                  0.9942                        36   \n",
       "0                   0.803                        37   \n",
       "0                    0.98                        38   \n",
       "0                  0.9988                        39   \n",
       "0                   0.958                        40   \n",
       "0              0.47293332                        41   \n",
       "0                  0.9912                        42   \n",
       "0                  0.9595                        43   \n",
       "0                0.646775                        44   \n",
       "0                 0.27346                        45   \n",
       "0                  0.9952                        46   \n",
       "0                  0.9942                        47   \n",
       "0                  0.8749                        48   \n",
       "0                  0.9948                        49   \n",
       "0                  0.8583                        50   \n",
       "0                  0.9765                        51   \n",
       "0                  0.8581                        52   \n",
       "0                 0.85845                        53   \n",
       "0                 0.88325                        54   \n",
       "0              0.51497996                        55   \n",
       "0                  0.8984                        56   \n",
       "\n",
       "  entities_jsl_origin_sentence  \\\n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "0                            0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  "
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.jsl.wip.clinical.rd').predict('HISTORY OF PRESENT ILLNESS: The patient is a 40-year-old white male who presents with a chief complaint of \"chest pain\". The patient is diabetic and has a prior history of coronary artery disease. The patient presents today stating that his chest pain started yesterday evening and has been somewhat intermittent. The severity of the pain has progressively increased. He describes the pain as a sharp and heavy pain which radiates to his neck & left arm. He ranks the pain a 7 on a scale of 1-10. He admits some shortness of breath & diaphoresis. He states that he has had nausea & 3 episodes of vomiting tonight. He denies any fever or chills. He admits prior episodes of similar pain prior to his PTCA in 1995. He states the pain is somewhat worse with walking and seems to be relieved with rest. There is no change in pain with positioning. He states that he took 3 nitroglycerin tablets sublingually over the past 1 hour, which he states has partially relieved his pain. The patient ranks his present pain a 4 on a scale of 1-10. The most recent episode of pain has lasted one-hour. The patient denies any history of recent surgery, head trauma, recent stroke, abnormal bleeding such as blood in urine or stool or nosebleed.', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MKweTjPNe-UO"
   },
   "source": [
    "# [Medical Named Entity Recognizer: ner.deid.biobert\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_biobert_en.html\t)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 627
    },
    "id": "11FLxrN5frSE",
    "outputId": "1e2f725d-45cb-484f-9e59-6528ffbb9880"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_deid_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-f638b3ba-5ee3-44da-86ee-8b2bbe802090\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_deid</th>\n",
       "      <th>entities_deid_class</th>\n",
       "      <th>entities_deid_confidence</th>\n",
       "      <th>entities_deid_origin_chunk</th>\n",
       "      <th>entities_deid_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.9984</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>approximately two years ago</td>\n",
       "      <td>DATE</td>\n",
       "      <td>0.684725</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>VA Hospital</td>\n",
       "      <td>LOCATION</td>\n",
       "      <td>0.86225</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>John Green</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.95955</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>2347165768</td>\n",
       "      <td>ID</td>\n",
       "      <td>0.8887</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Day Hospital</td>\n",
       "      <td>LOCATION</td>\n",
       "      <td>0.79885</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>02/04/2003</td>\n",
       "      <td>DATE</td>\n",
       "      <td>0.9785</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.9938</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Day Hospital</td>\n",
       "      <td>LOCATION</td>\n",
       "      <td>0.84325004</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-f638b3ba-5ee3-44da-86ee-8b2bbe802090')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-f638b3ba-5ee3-44da-86ee-8b2bbe802090 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-f638b3ba-5ee3-44da-86ee-8b2bbe802090');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-4a8c3ff8-c599-4b1c-b86b-b013986aefe7\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4a8c3ff8-c599-4b1c-b86b-b013986aefe7')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-4a8c3ff8-c599-4b1c-b86b-b013986aefe7 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   \n",
       "\n",
       "                 entities_deid entities_deid_class entities_deid_confidence  \\\n",
       "0                        Smith                NAME                   0.9984   \n",
       "0  approximately two years ago                DATE                 0.684725   \n",
       "0                  VA Hospital            LOCATION                  0.86225   \n",
       "0                   John Green                NAME                  0.95955   \n",
       "0                   2347165768                  ID                   0.8887   \n",
       "0                 Day Hospital            LOCATION                  0.79885   \n",
       "0                   02/04/2003                DATE                   0.9785   \n",
       "0                        Smith                NAME                   0.9938   \n",
       "0                 Day Hospital            LOCATION               0.84325004   \n",
       "\n",
       "  entities_deid_origin_chunk entities_deid_origin_sentence  \\\n",
       "0                          0                             0   \n",
       "0                          1                             0   \n",
       "0                          2                             0   \n",
       "0                          3                             0   \n",
       "0                          4                             0   \n",
       "0                          5                             0   \n",
       "0                          6                             0   \n",
       "0                          7                             0   \n",
       "0                          8                             0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.deid.biobert').predict('HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-year-old white male veteran with multiple comorbidities, who has a history of bladder cancer diagnosed approximately two years ago by the VA Hospital, Dr. John Green (2347165768). He underwent a resection there. He was to be admitted to the Day Hospital for cystectomy. He was seen in Urology Clinic and Radiology Clinic on 02/04/2003. HOSPITAL COURSE: Mr. Smith presented to the Day Hospital in anticipation for Urology surgery. On evaluation, EKG, echocardiogram was abnormal, a Cardiology consult was obtained. A cardiac adenosine stress MRI was then proceeded, same was positive for inducible ischemia, mild-to-moderate inferolateral subendocardial infarction with peri-infarct ischemia. In addition, inducible ischemia seen in the inferior lateral septum. Mr. Smith underwent a left heart catheterization, which revealed two vessel coronary artery disease. The RCA, proximal was 95% stenosed and the distal 80% stenosed. The mid LAD was 85% stenosed and the distal LAD was 85% stenosed. There was four Multi-Link Vision bare metal stents placed to decrease all four lesions to 0%. Following intervention, Mr. Smith was admitted to 7 Ardmore Tower under Cardiology Service under the direction of Dr. Hart. Mr. Smith had a noncomplicated post-intervention hospital course. He was stable for discharge home on 02/07/2003 with instructions to take Plavix daily for one month and Urology is aware of the same. ', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qlENj2_ZgLrc"
   },
   "source": [
    "# [Medical Named Entity Recognizer: ner.deid.augmented\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_deid_augmented_en.html\t)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 870
    },
    "id": "3Wxsxq83gS6z",
    "outputId": "42810b81-f146-4c58-e92c-7d47790d781f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_deid_augmented download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-7c470814-6bfc-44bc-a4b9-a84b8ca3b19a\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_deid</th>\n",
       "      <th>entities_deid_class</th>\n",
       "      <th>entities_deid_confidence</th>\n",
       "      <th>entities_deid_origin_chunk</th>\n",
       "      <th>entities_deid_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.9998</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>VA Hospital</td>\n",
       "      <td>LOCATION</td>\n",
       "      <td>0.98885</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>John Green</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.8888</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>2347165768</td>\n",
       "      <td>ID</td>\n",
       "      <td>0.9967</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Day Hospital</td>\n",
       "      <td>LOCATION</td>\n",
       "      <td>0.9866</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>02/04/2003</td>\n",
       "      <td>DATE</td>\n",
       "      <td>0.8058</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.9998</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Day Hospital</td>\n",
       "      <td>LOCATION</td>\n",
       "      <td>0.9197</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.9997</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.9995</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>7 Ardmore Tower</td>\n",
       "      <td>LOCATION</td>\n",
       "      <td>0.7206667</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Hart</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.9996</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>NAME</td>\n",
       "      <td>0.9998</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>02/07/2003</td>\n",
       "      <td>DATE</td>\n",
       "      <td>0.9995</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-7c470814-6bfc-44bc-a4b9-a84b8ca3b19a')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-7c470814-6bfc-44bc-a4b9-a84b8ca3b19a button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-7c470814-6bfc-44bc-a4b9-a84b8ca3b19a');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-7df24432-8e3e-4cc7-95ad-fecc576f6243\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-7df24432-8e3e-4cc7-95ad-fecc576f6243')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-7df24432-8e3e-4cc7-95ad-fecc576f6243 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document    entities_deid  \\\n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...            Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...      VA Hospital   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...       John Green   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...       2347165768   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...     Day Hospital   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...       02/04/2003   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...            Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...     Day Hospital   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...            Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...            Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...  7 Ardmore Tower   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...             Hart   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...            Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...       02/07/2003   \n",
       "\n",
       "  entities_deid_class entities_deid_confidence entities_deid_origin_chunk  \\\n",
       "0                NAME                   0.9998                          0   \n",
       "0            LOCATION                  0.98885                          1   \n",
       "0                NAME                   0.8888                          2   \n",
       "0                  ID                   0.9967                          3   \n",
       "0            LOCATION                   0.9866                          4   \n",
       "0                DATE                   0.8058                          5   \n",
       "0                NAME                   0.9998                          6   \n",
       "0            LOCATION                   0.9197                          7   \n",
       "0                NAME                   0.9997                          8   \n",
       "0                NAME                   0.9995                          9   \n",
       "0            LOCATION                0.7206667                         10   \n",
       "0                NAME                   0.9996                         11   \n",
       "0                NAME                   0.9998                         12   \n",
       "0                DATE                   0.9995                         13   \n",
       "\n",
       "  entities_deid_origin_sentence  \\\n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.deid.augmented').predict('HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-year-old white male veteran with multiple comorbidities, who has a history of bladder cancer diagnosed approximately two years ago by the VA Hospital, Dr. John Green (2347165768). He underwent a resection there. He was to be admitted to the Day Hospital for cystectomy. He was seen in Urology Clinic and Radiology Clinic on 02/04/2003. HOSPITAL COURSE: Mr. Smith presented to the Day Hospital in anticipation for Urology surgery. On evaluation, EKG, echocardiogram was abnormal, a Cardiology consult was obtained. A cardiac adenosine stress MRI was then proceeded, same was positive for inducible ischemia, mild-to-moderate inferolateral subendocardial infarction with peri-infarct ischemia. In addition, inducible ischemia seen in the inferior lateral septum. Mr. Smith underwent a left heart catheterization, which revealed two vessel coronary artery disease. The RCA, proximal was 95% stenosed and the distal 80% stenosed. The mid LAD was 85% stenosed and the distal LAD was 85% stenosed. There was four Multi-Link Vision bare metal stents placed to decrease all four lesions to 0%. Following intervention, Mr. Smith was admitted to 7 Ardmore Tower under Cardiology Service under the direction of Dr. Hart. Mr. Smith had a noncomplicated post-intervention hospital course. He was stable for discharge home on 02/07/2003 with instructions to take Plavix daily for one month and Urology is aware of the same. ', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "brYx-UPwgf-W"
   },
   "source": [
    "# [Medical Named Entity Recognizer: ner.deid.enriched\t\t\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_deid_enriched_en.html\t)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 773
    },
    "id": "4mvfvBwTgufS",
    "outputId": "8f1daa5f-142e-4848-fe90-ae11702a8cef"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_deid_enriched download started this may take some time.\n",
      "[OK!]\n",
      "embeddings_clinical download started this may take some time.\n",
      "Approximate size to download 1.6 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-c50493cd-d724-4276-bb90-be84ed58d11f\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_deid</th>\n",
       "      <th>entities_deid_class</th>\n",
       "      <th>entities_deid_confidence</th>\n",
       "      <th>entities_deid_origin_chunk</th>\n",
       "      <th>entities_deid_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>PATIENT</td>\n",
       "      <td>0.9995</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>VA Hospital</td>\n",
       "      <td>HOSPITAL</td>\n",
       "      <td>0.7745</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>John Green</td>\n",
       "      <td>DOCTOR</td>\n",
       "      <td>0.6661</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>(2347165768</td>\n",
       "      <td>PHONE</td>\n",
       "      <td>0.90604997</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>02/04/2003</td>\n",
       "      <td>DATE</td>\n",
       "      <td>0.9998</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>PATIENT</td>\n",
       "      <td>0.9975</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>PATIENT</td>\n",
       "      <td>0.9988</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>PATIENT</td>\n",
       "      <td>0.9985</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Ardmore Tower</td>\n",
       "      <td>PATIENT</td>\n",
       "      <td>0.34765</td>\n",
       "      <td>8</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Hart</td>\n",
       "      <td>DOCTOR</td>\n",
       "      <td>0.9918</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>PATIENT</td>\n",
       "      <td>0.9983</td>\n",
       "      <td>10</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>02/07/2003</td>\n",
       "      <td>DATE</td>\n",
       "      <td>1.0</td>\n",
       "      <td>11</td>\n",
       "      <td>0</td>\n",
       "      <td>[[0.09692134708166122, 0.531410813331604, -0.0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-c50493cd-d724-4276-bb90-be84ed58d11f')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-c50493cd-d724-4276-bb90-be84ed58d11f button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-c50493cd-d724-4276-bb90-be84ed58d11f');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-37e69fc4-302b-41c1-bd0c-274f4c9ca307\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-37e69fc4-302b-41c1-bd0c-274f4c9ca307')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-37e69fc4-302b-41c1-bd0c-274f4c9ca307 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  entities_deid  \\\n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...          Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...    VA Hospital   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...     John Green   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...    (2347165768   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...     02/04/2003   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...          Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...          Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...          Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...  Ardmore Tower   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...           Hart   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...          Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...     02/07/2003   \n",
       "\n",
       "  entities_deid_class entities_deid_confidence entities_deid_origin_chunk  \\\n",
       "0             PATIENT                   0.9995                          0   \n",
       "0            HOSPITAL                   0.7745                          1   \n",
       "0              DOCTOR                   0.6661                          2   \n",
       "0               PHONE               0.90604997                          3   \n",
       "0                DATE                   0.9998                          4   \n",
       "0             PATIENT                   0.9975                          5   \n",
       "0             PATIENT                   0.9988                          6   \n",
       "0             PATIENT                   0.9985                          7   \n",
       "0             PATIENT                  0.34765                          8   \n",
       "0              DOCTOR                   0.9918                          9   \n",
       "0             PATIENT                   0.9983                         10   \n",
       "0                DATE                      1.0                         11   \n",
       "\n",
       "  entities_deid_origin_sentence  \\\n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  \n",
       "0  [[0.09692134708166122, 0.531410813331604, -0.0...  "
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.deid.enriched').predict('HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-year-old white male veteran with multiple comorbidities, who has a history of bladder cancer diagnosed approximately two years ago by the VA Hospital, Dr. John Green (2347165768). He underwent a resection there. He was to be admitted to the Day Hospital for cystectomy. He was seen in Urology Clinic and Radiology Clinic on 02/04/2003. HOSPITAL COURSE: Mr. Smith presented to the Day Hospital in anticipation for Urology surgery. On evaluation, EKG, echocardiogram was abnormal, a Cardiology consult was obtained. A cardiac adenosine stress MRI was then proceeded, same was positive for inducible ischemia, mild-to-moderate inferolateral subendocardial infarction with peri-infarct ischemia. In addition, inducible ischemia seen in the inferior lateral septum. Mr. Smith underwent a left heart catheterization, which revealed two vessel coronary artery disease. The RCA, proximal was 95% stenosed and the distal 80% stenosed. The mid LAD was 85% stenosed and the distal LAD was 85% stenosed. There was four Multi-Link Vision bare metal stents placed to decrease all four lesions to 0%. Following intervention, Mr. Smith was admitted to 7 Ardmore Tower under Cardiology Service under the direction of Dr. Hart. Mr. Smith had a noncomplicated post-intervention hospital course. He was stable for discharge home on 02/07/2003 with instructions to take Plavix daily for one month and Urology is aware of the same. ', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WRNapAgMg_6u"
   },
   "source": [
    "# [Medical Named Entity Recognizer: ner.deid.enriched_biobert\t\t\t\t\t](https://nlp.johnsnowlabs.com/2021/04/01/ner_deid_enriched_biobert_en.html\t)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 578
    },
    "id": "ZeDVaiP4hILH",
    "outputId": "a1a054af-db93-4da5-aa1d-3dcc2fe360cb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_deid_enriched_biobert download started this may take some time.\n",
      "[OK!]\n",
      "biobert_pubmed_base_cased download started this may take some time.\n",
      "Approximate size to download 386.4 MB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-dffd339f-c1d9-4a9e-ab47-f649ed7df9de\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_deid</th>\n",
       "      <th>entities_deid_class</th>\n",
       "      <th>entities_deid_confidence</th>\n",
       "      <th>entities_deid_origin_chunk</th>\n",
       "      <th>entities_deid_origin_sentence</th>\n",
       "      <th>word_embedding_biobert</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>PATIENT</td>\n",
       "      <td>0.7029</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>VA Hospital</td>\n",
       "      <td>HOSPITAL</td>\n",
       "      <td>0.88594997</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>John Green</td>\n",
       "      <td>DOCTOR</td>\n",
       "      <td>0.89355004</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>2347165768</td>\n",
       "      <td>PHONE</td>\n",
       "      <td>0.7562</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Day Hospital</td>\n",
       "      <td>HOSPITAL</td>\n",
       "      <td>0.98115003</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>02/04/2003</td>\n",
       "      <td>DATE</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Smith</td>\n",
       "      <td>PATIENT</td>\n",
       "      <td>0.8063</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...</td>\n",
       "      <td>Day Hospital</td>\n",
       "      <td>HOSPITAL</td>\n",
       "      <td>0.9943</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-1.249812364578247, 0.44256749749183655, -0....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-dffd339f-c1d9-4a9e-ab47-f649ed7df9de')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-dffd339f-c1d9-4a9e-ab47-f649ed7df9de button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-dffd339f-c1d9-4a9e-ab47-f649ed7df9de');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-d16af7e9-316b-4a23-a954-09d0008412b0\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-d16af7e9-316b-4a23-a954-09d0008412b0')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-d16af7e9-316b-4a23-a954-09d0008412b0 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document entities_deid  \\\n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...         Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...   VA Hospital   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...    John Green   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...    2347165768   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...  Day Hospital   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...    02/04/2003   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...         Smith   \n",
       "0  HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-...  Day Hospital   \n",
       "\n",
       "  entities_deid_class entities_deid_confidence entities_deid_origin_chunk  \\\n",
       "0             PATIENT                   0.7029                          0   \n",
       "0            HOSPITAL               0.88594997                          1   \n",
       "0              DOCTOR               0.89355004                          2   \n",
       "0               PHONE                   0.7562                          3   \n",
       "0            HOSPITAL               0.98115003                          4   \n",
       "0                DATE                      1.0                          5   \n",
       "0             PATIENT                   0.8063                          6   \n",
       "0            HOSPITAL                   0.9943                          7   \n",
       "\n",
       "  entities_deid_origin_sentence  \\\n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "0                             0   \n",
       "\n",
       "                              word_embedding_biobert  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  \n",
       "0  [[-1.249812364578247, 0.44256749749183655, -0....  "
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.deid.enriched_biobert').predict('HISTORY OF PRESENT ILLNESS: Mr. Smith is a 60-year-old white male veteran with multiple comorbidities, who has a history of bladder cancer diagnosed approximately two years ago by the VA Hospital, Dr. John Green (2347165768). He underwent a resection there. He was to be admitted to the Day Hospital for cystectomy. He was seen in Urology Clinic and Radiology Clinic on 02/04/2003. HOSPITAL COURSE: Mr. Smith presented to the Day Hospital in anticipation for Urology surgery. On evaluation, EKG, echocardiogram was abnormal, a Cardiology consult was obtained. A cardiac adenosine stress MRI was then proceeded, same was positive for inducible ischemia, mild-to-moderate inferolateral subendocardial infarction with peri-infarct ischemia. In addition, inducible ischemia seen in the inferior lateral septum. Mr. Smith underwent a left heart catheterization, which revealed two vessel coronary artery disease. The RCA, proximal was 95% stenosed and the distal 80% stenosed. The mid LAD was 85% stenosed and the distal LAD was 85% stenosed. There was four Multi-Link Vision bare metal stents placed to decrease all four lesions to 0%. Following intervention, Mr. Smith was admitted to 7 Ardmore Tower under Cardiology Service under the direction of Dr. Hart. Mr. Smith had a noncomplicated post-intervention hospital course. He was stable for discharge home on 02/07/2003 with instructions to take Plavix daily for one month and Urology is aware of the same. ', output_level='chunk')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YiJaUP9xhWzG"
   },
   "source": [
    "# [Medical Named Entity Recognizer: ner.deid.dl\t\t](https://nlp.johnsnowlabs.com/2021/03/31/ner_deidentify_dl_en.html\t)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "ozuTOvFAhgWX",
    "outputId": "80a2d145-6757-41df-a2ff-d4cdf340eb48"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Warning::Spark Session already created, some configs may not take.\n",
      "Warning::Spark Session already created, some configs may not take.\n",
      "ner_wikiner_glove_840B_300 download started this may take some time.\n",
      "Approximate size to download 14.8 MB\n",
      "[OK!]\n",
      "glove_840B_300 download started this may take some time.\n",
      "Approximate size to download 2.3 GB\n",
      "[OK!]\n",
      "Warning::Spark Session already created, some configs may not take.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-0e3996f7-0dae-4053-9802-ff823786bba9\" class=\"colab-df-container\">\n",
       "    <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>document</th>\n",
       "      <th>entities_wikiner_glove_840B_300</th>\n",
       "      <th>entities_wikiner_glove_840B_300_class</th>\n",
       "      <th>entities_wikiner_glove_840B_300_confidence</th>\n",
       "      <th>entities_wikiner_glove_840B_300_origin_chunk</th>\n",
       "      <th>entities_wikiner_glove_840B_300_origin_sentence</th>\n",
       "      <th>word_embedding_glove</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Record date : 2093-01-13 , David Hale , M....</td>\n",
       "      <td>Record</td>\n",
       "      <td>ORG</td>\n",
       "      <td>0.3771</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Record date : 2093-01-13 , David Hale , M....</td>\n",
       "      <td>David Hale</td>\n",
       "      <td>PER</td>\n",
       "      <td>0.81305003</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Record date : 2093-01-13 , David Hale , M....</td>\n",
       "      <td>M.D . , Name : Hendrickson</td>\n",
       "      <td>MISC</td>\n",
       "      <td>0.6513</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Record date : 2093-01-13 , David Hale , M....</td>\n",
       "      <td>Ora MR</td>\n",
       "      <td>PER</td>\n",
       "      <td>0.53705</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Record date : 2093-01-13 , David Hale , M....</td>\n",
       "      <td>PCP : Oliveira</td>\n",
       "      <td>MISC</td>\n",
       "      <td>0.54356664</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Record date : 2093-01-13 , David Hale , M....</td>\n",
       "      <td>Cocke County Baptist Hospital</td>\n",
       "      <td>LOC</td>\n",
       "      <td>0.333825</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>A . Record date : 2093-01-13 , David Hale , M....</td>\n",
       "      <td>0295 Keats Street</td>\n",
       "      <td>LOC</td>\n",
       "      <td>0.49306667</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>[[-0.2099599987268448, -0.15577000379562378, -...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>\n",
       "    <div class=\"colab-df-buttons\">\n",
       "\n",
       "  <div class=\"colab-df-container\">\n",
       "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0e3996f7-0dae-4053-9802-ff823786bba9')\"\n",
       "            title=\"Convert this dataframe to an interactive table.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
       "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
       "  </svg>\n",
       "    </button>\n",
       "\n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    .colab-df-buttons div {\n",
       "      margin-bottom: 4px;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "    <script>\n",
       "      const buttonEl =\n",
       "        document.querySelector('#df-0e3996f7-0dae-4053-9802-ff823786bba9 button.colab-df-convert');\n",
       "      buttonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "      async function convertToInteractive(key) {\n",
       "        const element = document.querySelector('#df-0e3996f7-0dae-4053-9802-ff823786bba9');\n",
       "        const dataTable =\n",
       "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                    [key], {});\n",
       "        if (!dataTable) return;\n",
       "\n",
       "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "          + ' to learn more about interactive tables.';\n",
       "        element.innerHTML = '';\n",
       "        dataTable['output_type'] = 'display_data';\n",
       "        await google.colab.output.renderOutput(dataTable, element);\n",
       "        const docLink = document.createElement('div');\n",
       "        docLink.innerHTML = docLinkHtml;\n",
       "        element.appendChild(docLink);\n",
       "      }\n",
       "    </script>\n",
       "  </div>\n",
       "\n",
       "\n",
       "<div id=\"df-9dfb1235-8e1d-486f-8caf-e46f1a940ac3\">\n",
       "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-9dfb1235-8e1d-486f-8caf-e46f1a940ac3')\"\n",
       "            title=\"Suggest charts.\"\n",
       "            style=\"display:none;\">\n",
       "\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "     width=\"24px\">\n",
       "    <g>\n",
       "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
       "    </g>\n",
       "</svg>\n",
       "  </button>\n",
       "\n",
       "<style>\n",
       "  .colab-df-quickchart {\n",
       "      --bg-color: #E8F0FE;\n",
       "      --fill-color: #1967D2;\n",
       "      --hover-bg-color: #E2EBFA;\n",
       "      --hover-fill-color: #174EA6;\n",
       "      --disabled-fill-color: #AAA;\n",
       "      --disabled-bg-color: #DDD;\n",
       "  }\n",
       "\n",
       "  [theme=dark] .colab-df-quickchart {\n",
       "      --bg-color: #3B4455;\n",
       "      --fill-color: #D2E3FC;\n",
       "      --hover-bg-color: #434B5C;\n",
       "      --hover-fill-color: #FFFFFF;\n",
       "      --disabled-bg-color: #3B4455;\n",
       "      --disabled-fill-color: #666;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart {\n",
       "    background-color: var(--bg-color);\n",
       "    border: none;\n",
       "    border-radius: 50%;\n",
       "    cursor: pointer;\n",
       "    display: none;\n",
       "    fill: var(--fill-color);\n",
       "    height: 32px;\n",
       "    padding: 0;\n",
       "    width: 32px;\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart:hover {\n",
       "    background-color: var(--hover-bg-color);\n",
       "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "    fill: var(--button-hover-fill-color);\n",
       "  }\n",
       "\n",
       "  .colab-df-quickchart-complete:disabled,\n",
       "  .colab-df-quickchart-complete:disabled:hover {\n",
       "    background-color: var(--disabled-bg-color);\n",
       "    fill: var(--disabled-fill-color);\n",
       "    box-shadow: none;\n",
       "  }\n",
       "\n",
       "  .colab-df-spinner {\n",
       "    border: 2px solid var(--fill-color);\n",
       "    border-color: transparent;\n",
       "    border-bottom-color: var(--fill-color);\n",
       "    animation:\n",
       "      spin 1s steps(1) infinite;\n",
       "  }\n",
       "\n",
       "  @keyframes spin {\n",
       "    0% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "      border-left-color: var(--fill-color);\n",
       "    }\n",
       "    20% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    30% {\n",
       "      border-color: transparent;\n",
       "      border-left-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    40% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-top-color: var(--fill-color);\n",
       "    }\n",
       "    60% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "    }\n",
       "    80% {\n",
       "      border-color: transparent;\n",
       "      border-right-color: var(--fill-color);\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "    90% {\n",
       "      border-color: transparent;\n",
       "      border-bottom-color: var(--fill-color);\n",
       "    }\n",
       "  }\n",
       "</style>\n",
       "\n",
       "  <script>\n",
       "    async function quickchart(key) {\n",
       "      const quickchartButtonEl =\n",
       "        document.querySelector('#' + key + ' button');\n",
       "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
       "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
       "      try {\n",
       "        const charts = await google.colab.kernel.invokeFunction(\n",
       "            'suggestCharts', [key], {});\n",
       "      } catch (error) {\n",
       "        console.error('Error during call to suggestCharts:', error);\n",
       "      }\n",
       "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
       "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
       "    }\n",
       "    (() => {\n",
       "      let quickchartButtonEl =\n",
       "        document.querySelector('#df-9dfb1235-8e1d-486f-8caf-e46f1a940ac3 button');\n",
       "      quickchartButtonEl.style.display =\n",
       "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "    })();\n",
       "  </script>\n",
       "</div>\n",
       "    </div>\n",
       "  </div>\n"
      ],
      "text/plain": [
       "                                            document  \\\n",
       "0  A . Record date : 2093-01-13 , David Hale , M....   \n",
       "0  A . Record date : 2093-01-13 , David Hale , M....   \n",
       "0  A . Record date : 2093-01-13 , David Hale , M....   \n",
       "0  A . Record date : 2093-01-13 , David Hale , M....   \n",
       "0  A . Record date : 2093-01-13 , David Hale , M....   \n",
       "0  A . Record date : 2093-01-13 , David Hale , M....   \n",
       "0  A . Record date : 2093-01-13 , David Hale , M....   \n",
       "\n",
       "  entities_wikiner_glove_840B_300 entities_wikiner_glove_840B_300_class  \\\n",
       "0                          Record                                   ORG   \n",
       "0                      David Hale                                   PER   \n",
       "0      M.D . , Name : Hendrickson                                  MISC   \n",
       "0                          Ora MR                                   PER   \n",
       "0                  PCP : Oliveira                                  MISC   \n",
       "0   Cocke County Baptist Hospital                                   LOC   \n",
       "0               0295 Keats Street                                   LOC   \n",
       "\n",
       "  entities_wikiner_glove_840B_300_confidence  \\\n",
       "0                                     0.3771   \n",
       "0                                 0.81305003   \n",
       "0                                     0.6513   \n",
       "0                                    0.53705   \n",
       "0                                 0.54356664   \n",
       "0                                   0.333825   \n",
       "0                                 0.49306667   \n",
       "\n",
       "  entities_wikiner_glove_840B_300_origin_chunk  \\\n",
       "0                                            0   \n",
       "0                                            1   \n",
       "0                                            2   \n",
       "0                                            3   \n",
       "0                                            4   \n",
       "0                                            5   \n",
       "0                                            6   \n",
       "\n",
       "  entities_wikiner_glove_840B_300_origin_sentence  \\\n",
       "0                                               0   \n",
       "0                                               0   \n",
       "0                                               0   \n",
       "0                                               0   \n",
       "0                                               0   \n",
       "0                                               0   \n",
       "0                                               0   \n",
       "\n",
       "                                word_embedding_glove  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  \n",
       "0  [[-0.2099599987268448, -0.15577000379562378, -...  "
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nlp.load('en.med_ner.deid.dl').predict('A . Record date : 2093-01-13 , David Hale , M.D . , Name : Hendrickson , Ora MR . # 7194334 Date : 01/13/93 PCP : Oliveira , 25 month years-old , Record date : 2079-11-09 . Cocke County Baptist Hospital . 0295 Keats Street ', output_level='chunk')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9X0WomcAFNXG"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
